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Abstract 

Several  previous  studies  investigated  the  performance  of  competing  circular  error 
probable  (CEP)  estimators  for  small  samples.  This  estimation  is  important  in  ICBM 
analysis  because,  due  to  expense,  there  are  a  limited  number  of  ICBM  test  launches.  In 
the  most  recent  previous  study  (1993),  Tongue  considered  five  CEP  estimators  in  a 
simulation  test,  attempting  to  determine  the  behavior  of  these  estimators  for  populations 
of  various  bias,  ellipticity,  correlation,  and  sample  size.  In  this  paper,  we  build  on 
Tongue’s  findings  in  three  ways; 

1 . )  The  number  of  estimators  compared  is  expanded  to  eight. 

2. )  Different  factors  and  factor  levels  are  used. 

3. )  In  addition  to  analyzing  simulated  samples  based  on  the  population  parameters 
used  to  create  them,  we  sort  the  entire  set  of  samples  generated  into  subsets  (sample 
analysis  sets)  based  strictly  on  their  sample  statistics.  Analysis  conducted  on  these  sample 
analysis  sets  models  the  real  life  situation  of  estimating  CEP,  given  a  small  sample  from  an 
unknown  population,  using  only  sample  statistics. 


IX 


L  INTRODUCTION 


Circular  error  probable,  or  CEP,  is  a  statistical  parameter  used  to  describe  the 
accuracy  of  ballistic  projectiles.  This  widely  used  performance  measure  has  a  major 
impact  on  operational  planning  and  analysis,  as  demonstrated  by  these  three  diverse 
examples: 

1.  The  CEP  of  weapon  systems  such  as  ICBMs  and  the  HELLFIRE  antitank 
missile  affects  operational  planning  [Lewis  (1994)]. 

2.  CEP  is  used  to  describe  the  accuracy  of  surface  to  air  missiles  in  the  Joint 
Munitions  Effectiveness  Manual  (JMEM)  [Asner(1993)]. 

3.  CEP  influences  weapon  system  performance  in  probabilistic  combat 
simulation  models. 

The  primary  purpose  of  this  thesis  is  to  determine  through  a  comparison  study  the 
effectiveness  of  a  number  of  CEP  estimators  for  small  samples  under  various  conditions. 
This  determination  is  vital  to  intercontinental  ballistic  missile  (ICBM)  analysis  since  only  a 
small  number  of  test  launches  are  conducted  annually  due  to  their  great  expense  [Ethridge 
(1983),  1].  Furthermore,  updates  to  ICBM  components  such  as  the  missile  guidance 
system  (MGS)  render  flight  test  data  which  was  collected  prior  to  the  update  non- 
applicable.  Therefore,  analysis  of  ICBM  test  launches  typically  involves  very  small  sets  of 
data. 

In  this  introductory  chapter,  we  first  present  a  general  overview  of  basic  CEP 
analysis  concepts  and  terminology.  We  next  discuss  previous  CEP  estimator  comparison 
studies.  The  chapter  concludes  with  an  outline  of  our  experimental  objectives  and  an 
overview  of  our  experimental  procedure. 
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Appendix  A  provides  a  key  for  the  notation  used  in  this  thesis.  It  also  lists  the 
formulas  used  for  calculating  common  sample  statistics  such  as  bias,  ellipticity,  and 
correlation. 

1.1  An  Overview  of  CEP  Analysis 

CEP  is  generally  defined  in  relation  to  either  the  target  or  the  mean  point  of 
impact.  CEP  defined  with  respect  to  the  target  is  the  radius  of  a  circle,  centered  on  the 
target,  such  that  the  probability  of  an  impact  landing  inside  the  circle  is  50%  [Nelson 
(1988),  3].  CEP  defined  with  respect  to  the  mean  point  of  impact  is  the  radius  of  a  circle, 
centered  on  the  mean  point  of  impact,  such  that  the  probability  of  an  impact  landing  inside 
the  circle  is  50%  [Thompson  (1964),  1].  To  distinguish  between  these  two,  we  use  "CEP" 
to  refer  to  CEP  with  respect  to  the  target  while  "CEPmpi"  refers  to  CEP  with  respect  to 
the  mean  point  of  impact.  In  this  study,  the  focus  is  on  CEP,  not  CEPmpi  . 

The  objective  in  CEP  analysis  is  to  determine  a  reliable  CEP  estimate  given  a  set  of 
projectile  impact  points  around  a  target.  These  ballistic  impacts  can  be  described  using  the 
following  Cartesian  coordinate  system  [Tongue  (1993),1-1  and  1-2]: 

1.  The  origin  (0,0)  corresponds  to  the  target. 

2.  The  crossrange  miss  distance  (error)  is  measured  on  the  X-axis  (crossrange 

error  axis). 

3.  The  downrange  miss  distance  (error)  is  measured  on  the  Y-axis  (downrange 

error  axis). 

4.  The  radial  miss  distance  n  for  a  point  (xj,  y,)  is  defined  by 

ri=V^i^+yi^  (1-1) 
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This  Cartesian  coordinate  system  is  described  in  Figure  1.1  below: 


incoming  projectile  /  . 

actual  target  point 


Figure  1.1  The  Cartesian  Coordinate  System  Used  in  CEP  Analysis 


The  mean  point  of  impact  (x,y  )  is  simply  the  coordinate  of  the  sample  crossrange  mean 
and  downrange  mean. 

For  most  CEP  estimators,  it  is  assumed  that  the  downrange  and  crossrange  miss 
distances  follow  a  bivariate  normal  distribution.  If  the  population  parameters  is  known, 
the  joint  probability  density  function  for  this  distribution  is: 


f(x,y)  = 


271  CTxCTy^/  1-p^ 


[2(l-p2)J 

e 


X-M.X 


f(x-t^x)(y-^^y)l 

P  [  axCTy 


y-K 


V  '"y 


(1-2) 


Since  this  distribution  forms  a  three  dimensional  shape,  it  is  often  described  using 
"contour  lines."  Figure  1.2  portrays  the  contour  lines  corresponding  to  a  given  bivariate 
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normal  distribution.  Note  that  each  contour  line  circumscribes  a  given  proportion  of  the 
impact  score  population: 


Figure  1.2  A  Bivariate  Normal  Shape  and  its  Resulting  Contour  Lines 


Based  on  the  assumption  that  the  crossrange  and  downrange  impact  points  follow 
a  joint  bivariate  normal  density  function,  if  the  population  parameters  are  known,  CEP  is 
simply  a  circle  of  radius  r  such  that  the  double  integral  below  has  a  value  of  0.5: 


(1-3) 


Unfortunately,  this  integral  has  a  closed  form  solution  only  under  the  special 
conditions  described  in  Section  2.2.  For  the  cases  where  the  integral  has  no  closed  form 
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solution,  the  analyst  has  three  choices: 

1.  Approximate  the  integral  numerically  to  find  this  value  for  r. 

2.  Develop  an  empirical  formula  for  estimating  CEP  such  that  when  this  CEP  estimate  is 
substituted  in  for  r,  p(r)  will  be  approximately  0.5. 

3.  Apply  a  CEP  estimator  which  is  not  based  on  the  assumption  that  the  impact  points 
follow  a  bivariate  normal  distribution. 

Chapter  2  provides  an  in-depth  discussion  of  the  CEP  estimation  methods  considered  in 
this  thesis. 

1.2  Previous  CEP  Comparison  Studies 

In  this  section  we  overview  previous  comparison  studies  which  examined  the 
performance  of  CEP  estimators  for  "small"  samples  (sample  size  of  30  or  less)  under 
various  conditions. 

Elder  (1986)  presented  one  of  the  first  comprehensive  comparison  studies  of  CEP 
estimators.  Elder's  experiment  considered  only  bias  and  ellipticity.  Although  correlation 
can  significantly  influence  the  shape  of  the  bivariate  normal  surface.  Elder  did  not  include 
it  as  a  factor.  In  addition.  Elder  did  not  address  the  effect  of  sample  size  on  the 
performance  of  the  CEP  estimators  he  compared. 

Puhek's  1992  CEP  estimator  comparison  study  experiment  expanded  the  factors  to 
include  bias,  ellipticity,  correlation,  and  sample  size,  but  was  limited  in  four  ways: 

i. )  Only  four  CEP  estimators  were  compared. 

ii. )  Only  positive  correlation  values  were  used. 

iii. )  Estimate  variance  was  not  analyzed. 
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iv.)  No  replications  were  performed  at  any  of  the  design  points. 

Tongue's  1993  comparison  study  is  the  most  recent  and  the  most  complete.  Like 
Puhek,  the  four  factors  of  bias,  ellipticity,  correlation,  and  sample  size  were  used  in  the 
comparison  study.  Like  Puhek,  only  positive  correlation  values  were  considered  and  no 
replications  were  performed  at  any  of  the  design  points.  Tongue's  analysis  in  the 
comparison  of  five  CEP  estimators  involved  both  the  relative  error  and  the  variance  of  the 
estimates. 

In  this  thesis,  we  build  on  the  foundation  of  these  earlier  results  by  Elder  (1986), 
Puhek  (1992),  and  Tongue  (1993).  Six  of  the  eight  CEP  estimators  compared,  as  well  as 
three  of  the  measures  of  effectiveness  used  to  compare  the  CEP  estimators,  come  directly 
from  Tongue's  study. 

1,3  Problem  Statement 

In  this  paper,  the  results  of  Elder  (1986),  Puhek  (1992),  and  Tongue  (1993)  are 
extended  in  the  following  two  ways: 

1.  Like  the  three  previous  studies,  we  conduct  a  simulation  experiment  in  which  CEP 
estimators  are  compared  using  simulated  small  samples  fi'om  populations  whose 
parameters  are  known.  Our  experimental  design,  however,  includes  some  different  factors 
and  some  different  factor  levels  than  those  used  by  Elder,  Puhek,  or  Tongue. 

2.  The  three  previous  CEP  estimator  comparison  studies  developed  recommendations  for 
choosing  a  CEP  estimation  technique  that  required  the  analyst  to  know  the  values  of  the 
impact  population  parameters.  But  consider  the  example  of  ICBM  missile  accuracy 
analyst  -  the  impact  population  parameters  are  unknown!  In  this  study,  we  attempt  to 
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determine  the  CEP  estimator(s)  of  choice  based  strictly  on  sample,  rather  than  population, 
parameters.  By  basing  conclusions  entirely  on  sample  parameters,  the  real-life  situation 
faced  by  an  ICBM  accuracy  analyst  is  mirrored. 

1.4  Experimental  Overview 

The  term  "design  point'  is  used  in  this  thesis  to  refer  to  a  specific  combination  of 
levels  for  the  experimental  factors.  In  the  simulation  experiment,  we  first  select  relevant 
factors  and  then  select  levels  for  each  of  these  factors  to  determine  our  design  points. 

The  thesis  experiment  can  be  described  as  a  four  step  process; 

1.  Ten  simulated  samples  are  generated  at  each  design  point.  These  replications  at  every 
design  point  make  this  thesis  unique;  previous  comparison  studies  had  no  replications  at 
any  of  the  design  points. 

2.  The  eight  CEP  estimators  being  compared  in  this  thesis  are  applied  to  the  simulated 
output  fi’om  each  design  point.  These  resulting  CEP  estimates  are  compared  to  the  true 
CEP  for  the  population  from  which  the  sample  was  generated.  Bear  in  mind  that  in  each 
case  the  parameters  of  the  underlying  bivariate  normal  distribution  are  determined  by  the 
design  point  and  are  thus  known. 

3.  All  of  the  sample  data  generated  is  next  categorized  into  subsets  (sample  analysis  sets) 
based  on  the  sample  statistics  of  sample  size,  sample  bias,  sample  correlation,  and  the 
sample  downrange  to  crossrange  standard  deviation  ratio  (Sx  /Sy).  These  statistics  are 
defined  in  Appendix  A. 

4.  The  eight  CEP  estimators  considered  in  the  thesis  experiment  are  appUed  to  each 
sample  analysis  set  and  the  results  are  analyzed.  When  we  analyzed  the  sample  analysis 


7 


sets,  the  underlying  bivariate  normal  distribution(s)  for  the  data  in  each  set  was  unknown! 
This  comparison  of  CEP  estimators  based  solely  on  sample  statistics  is  unique  to  this 
study, 

1.5  Obiectives 

The  three  fundamental  objectives  of  our  simulation  experiment  were  to: 

1 .  Provide  a  method  to  determine  which  CEP  estimator(s)  to  use  for  a  given  particular  set 
of  conditions  for  a  small  sample  (15  or  less). 

2.  Compare  the  simulation  experiment  results  based  on  the  design  points,  where  the 
population  parameters  are  known,  with  the  simulation  experiment  results  based  on  the 
sample  analysis  sets,  where  the  population  parameters  are  unknown. 

3.  Verify  the  results  from  earlier  CEP  estimator  comparison  studies. 

The  remaining  chapters  all  describe  some  aspect  of  the  thesis  experiment  used  to 
meet  these  three  objectives.  A  reference  for  the  mathematical  formulas  used  for 
calculating  each  CEP  estimator  considered  in  the  experiment  is  provided  in  Chapter  2.  In 
Chapter  3  we  develop  the  experimental  approach,  while  in  Chapter  4  we  describe  the 
experimental  results.  Based  on  these  results,  we  present  recommendations  in  Chapter  5. 
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n.  CEP  ESTIMATORS 


This  chapter  briefly  reviews  the  development  of  CEP  estimation  methods,  presents 
a  system  for  categorizing  these  methods,  and  describes  the  eight  CEP  estimators 
considered  in  this  thesis  in  detail. 

2.1  Back2round 

One  of  the  earliest  CEP  estimation  aids  was  the  publication  of  Offset  Circle 
Probabilities  by  the  Rand  Corporation  [Rand  R-234  (1952)].  R-234  consists  of  a  set  of 
lookup  tables  which  allow  for  the  estimation  of  CEP  based  upon  CEPmpi  and  the  sample 
bias  (defined  in  Appendix  A).  This  set  of  tables  provided  an  alternative  to  the  tedious 
calculations  involved  in  estimating  CEP  "by  hand"  in  the  pre-computer  era.  Pesapane  and 
Irvine  later  developed  the  modified  Rand-234  CEP  estimator  by  fitting  a  wide  range  of 
values  from  these  tables  to  a  cubic  polynomial  [Pesapane  and  Irvine  (1977)].  This 
estimator  is  not  valid  for  certain  cases  of  high  ellipticity  or  high  bias.  The  modified  Rand 
R-234  CEP  estimation  method  is  described  in  detail  in  Section  2.3  of  this  thesis. 

Harter  (1960)  and  Kamat  (1962)  published  early  papers  which  contained  algebraic 
CEP  estimation  techniques.  The  CEP  estimators  from  Kamat's  study  assumed  that  the 
crossrahge  and  downrange  values  followed  a  perfectly  circular  joint  bivariate  normal 
distribution  centered  at  (0,0).  This  type  of  distribution  implies  that; 

1.  Crossrange  and  downrange  means  both  equal  zero. 

2.  Crossrange  and  downrange  standard  deviations  are  equal. 

3.  No  correlation  exists  between  the  crossrange  and  downrange  miss  distances. 
Harter  allowed  for  ellipticity  in  his  lookup  tables,  but  maintained  the  assumption  that  the 
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bivariate  normal  distribution  was  centered  around  the  target  point.  Unfortunately,  in  the 
study  of  actual  projectiles,  the  distribution  of  impacts  is  often  neither  centered  at  the  target 
point  nor  perfectly  circular. 

Grubbs'  publication  of  Approximate  Circular  and  Noncircular  Offset  Probabilities 
of  Hitting  [1964]  spawned  a  number  of  CEP  estimators.  Two  of  the  most  common  are 
the  Grubbs-Patniak  chi-square  CEP  estimator,  which  is  described  in  Section  2.3,  and  the 
Grubbs-Patniak  /  Wilson-Hilferty  CEP  estimator.  Elder  [1986]  cited  that  Strategic  Air 
Command  ICBM  accuracy  analysts  relied  on  either  one  of  these  two  Grubbs  estimators  or 
the  modified  Rand  R-234  estimator  for  CEP  calculations.  Both  of  the  Grubbs  estimators 
are  similarly  calculated;  they  simply  use  different  methods  to  calculate  the  inverse  chi- 
square  component  in  their  algebraic  formulas. 

Ethridge  [1983]  introduced  the  robust,  unbiased  CEP  estimator  described  in 
Section  2.3.  Ethridge's  estimator  is  unique  in  the  respect  that  it  is  not  based  on  the 
assumption  that  the  impacts  follow  a  joint  bivariate  normal  distribution. 

As  computers  and  computer  software  developed  in  the  1980's,  it  became  possible 
to  numerically  integrate  the  assumed  bivariate  normal  distribution  of  balUstic  impacts  in  a 
relatively  short  amount  of  time.  Two  examples  of  numerical  CEP  estimators  are  the  direct 
numerical  integration  method  (referred  to  as  the  "exact"  method  in  some  papers)  and  the 
correlated  bivariate  normal  (CBN)  method. 

Tongue  (1993)  empirically  derived  an  algebraic  formula  for  predicting  the  relative 
error  value  when  the  CEP  is  estimated  using  the  correlated  bivariate  normal  (CBN) 
numerical  integration  CEP  estimator.  Tongue's  preliminary  analysis  indicated  that  his 
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modified  CBN  estimator,  described  in  Section  2.3,  generally  gave  better  estimates  than 
the  numerical  CBN  estimator  [Tongue  (1993),  page  5-19],  If  so,  additional  modifications 
of  existing  CEP  estimators  could  be  similarly  developed. 

2.2  CEP  Estimator  Categories 

Smith  (1982)  classified  CEP  estimation  methods  into  the  five  sets  we  use  to 
categorize  the  CEP  estimators  considered  in  this  thesis.  We  next  describe  each  of  these 
groups; 

1 .  Nonparcanetric  Methods:  These  CEP  estimators  make  no  assumptions  regarding  the 
underlying  population  of  ballistic  impacts.  Smith  uses  the  sample  median  as  an  example  of 
a  nonparametric  CEP  estimator. 

Unlike  nonparametric  CEP  estimators,  the  remaining  four  categories  are  based  on 
the  underlying  assumption  that  the  impacts  follow  some  sort  of  theoretical  joint  probability 
distribution,  usually  the  joint  bivariate  normal  distribution. 

2.  Closed-Form  Integration  of  the  Joint  Bivariate  Normal  Density  Function:  Closed 
form  integration  of  the  joint  bivariate  normal  density  function  is  only  possible  when  the 
distribution  is  assumed  be  perfectly  circular  and  centered  at  the  origin.  As  previously 
stated,  these  conditions  are  often  not  met  in  the  analysis  of  actual  ballistic  impacts.  If 
these  conditions  are  met,  and  a  represents  the  equal  crossrange  and  downrange  standard 
deviation  values  of  the  unbiased  circular  bivariate  normal  distribution,  CEP  is  estimated 
by: 

CEP  =  1.1774a.  (2-1) 

3 .  Algebraic  Approximation  CEP  Estimation  Methods:  The  majority  of  CEP  estimators 
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fall  into  this  category.  These  approximation  methods  use  an  algebraic  formula  of  impact 
population  parameters  (or  their  corresponding  sample  statistics)  to  estimate  CEP. 

4.  Numerical  Integration  of  the  Joint  Bivariate  Normal  Density  Function:  Unlike 
closed-form  integration,  numerical  integration  approximation  techniques  allow  for 
correlation,  bias,  and  ellipticity  [Elder  (1986),  1.2].  If  these  population  values  are  known, 
numerical  integration  provides  the  most  accurate  approximation  of  CEP.  When  only 
sample  statistics  are  known,  these  approximation  methods  can  be  used  as  estimators. 

The  author  verified  in  a  comparison  of  three  different  numerical  integration 
methods  (the  correlated  bivariate  normal  method,  the  Taylor  series  expansion  method,  and 
the  direct  numerical  approximation  method)  that  different  numerical  integration  methods 
produce  essentially  identical  results. 

5.  Monte  Carlo  Sampling  Methods:  These  methods  are  used  to  calculate  the  probability 
of  an  impact  landing  inside  a  circle  of  a  known  radius.  These  methods,  as  described  by 
Tongue  (1993),  essentially  estimate  CEP  using  simulation.  Impact  population  parameters 
are  estimated  using  sample  statistics,  and  a  set  of  "impacts"  is  simulated  based  on  these 
parameters.  An  initial  guess  for  the  CEP  value  is  determined,  using  for  example  an 
algebraic  estimator.  If  half  of  the  simulated  impacts  lie  in  a  circle  centered  at  the  target 
with  a  radius  equal  to  the  CEP  guess,  the  radius  of  this  circle  becomes  the  CEP  estimate. 
If  fewer  than  half  land  inside  this  circle,  the  CEP  guess  is  increased  by  some  increment  and 
the  process  is  repeated;  if  more  than  half  of  the  simulated  impacts  land  inside  the  circle, 
the  CEP  guess  is  decreased  by  some  increment  and  the  process  repeated. 

Smith  (1982)  concluded  that  for  small  samples  Monte  Carlo  sampling  techniques 
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are  good  tools  for  evaluating  other  CEP  estimators,  but  impractical  tools  for  actually 
estimating  CEP  because  they  use  excessive  computer  calculation  time  [Smith  (1982),  3], 
Two  additional  limitations  exist  when  trying  to  dpply  these  techniques  to  small  samples: 

1 .  The  sample  estimates  for  the  population  parameters  are  not  reliable  for  small  samples. 

2.  For  very  small  sample  sizes,  wide  bands  could  exist  between  the  points  near  the 
approximated  CEP  radius  for  the  simulated  samples. 

2.3  Description  of  the  CEP  Estimators  Compared  in  This  Study 

In  the  experimental  phase  of  this  research,  eight  individual  CEP  estimators  were 
evaluated.  We  now  describe  how  to  calculate  each  of  these  estimators  and  demonstrate 
that  they  can  provide  different  CEP  estimates  for  the  same  data  set.  The  practicing  CEP 
analyst  can  use  this  section  as  a  reference  for  CEP  estimation  methods. 

1.  The  Sample  Median  CEP  Estimator:  Smith  (1982)  claimed  that  nonparametric  CEP 
estimation  methods  are  not  suited  for  small  samples.  This  assertion  was  tested  by 
including  the  sample  median  of  the  radial  miss  distances  in  the  thesis  experiment. 

The  sample  median  CEP  estimation  value  for  a  sample  of  size  n,  which  we  denote 
as  Smed,  is  found  by  first  sequentially  ordering  the  radial  miss  distances  from  smallest  to 
largest.  Let  qij  to  r[„]  represent  these  ordered  values.  Smed  is  then  found  as  follows: 


Smed=  {  r 


n  +  1 


if  n  is  odd,  — 


rr„i  +  rr 


[I*-] 


if  n  is  even} 


(2-2) 


2.  The  Ethridge  CEP  Estimator:  As  mentioned  in  Section  1,  Ethridge  (1983)  developed 
an  algebraic  estimator  which  did  not  use  the  underlying  assumption  that  the  impact 
population  follows  some  bivariate  normal  distribution.  Ethridge  (1983)  instead  assumed 
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that  the  square  root  of  the  radial  miss  distances  followed  the  logarithmic  generalized 
exponential  power  distribution. 

For  a  given  set  of  impact  coordinates,  Ethridge  incorporated  the  sample  kurtosis 
and  sample  median  statistics  into  his  CEP  estimator.  To  develop  Ethridge's  formula,  we 
first  define  h  and  t . 


ti  =  In  (ri)  ==  In  ( )  for  each  sample  point  (xi ,  yO 
Then  average  these  ti  values  to  obtain  t : 


1  " 

t  =  -yti 


Sample  kurtosis  k  is  then  computed  by: 


k=  _»!_ 
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1=1 


Next,  let  St  ^  equal  the  sample  variance  of  the  ti  values,  computed  by: 


(2-3) 


(2-4) 


(2-5) 


1  =  1 


(2-6) 


Hogg's  unbiased  estimator  for  the  mean,  which  uses  "weighted"  values,  is  used  to 
estimate  the  mean  of  the  values.  For  each  coordinate  in  the  sample,  dt  is  defined  for 
computing  "weight"  w, . 

di  =  maximum  { 1  -[  0.03(k  -  3  )^  (ti  -  Smedf  (st)'^  ],  0.01  }  (2-7) 


Wi  = 


J_ 


(2-8) 
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These  Wi  values  are  then  used  to  compute  Hogg's  estimator  u: 


U=^  Wit;  (2-9) 

i=l 

Finally,  Ethridge's  CEP  estimator,  referred  to  hereafter  simply  as  Ethridge  in  this 
study ,  is  defined  by; 

Ethridge  =  e“  (2- 1 0) 

It  should  be  noted  that  the  simulated  sample  values  that  are  generated  in  the  thesis 
experiment  are  all  drawn  from  some  type  of  bivariate  normal  distribution,  even  though 
Ethridge  and  the  nonparametric  Smed  do  not  require  this  assumption. 

3.  The  Modified  Rand  R-234  CEP  Estimator.  The  algebraic  Modified  Rand  R-234  CEP 
estimation  formula  is  the  result  of  a  cubic  polynomial  regression  analysis  of  the  CEP  table 
values  found  in  the  Rand  R-234  paper  described  in  Section  2. 1  [Pesapane  and  Irvine 
(1977)]. 

Before  presenting  the  modified  Rand  R-234  CEP  estimator,  we  first  define  it's 
components.  For  any  bivariate  normal  distribution,  the  crossrange  and  downrange  axes 
can  be  rotated  to  form  two  new  axes  such  that  the  correlation  of  the  distribution  is  zero  in 
the  newly  formed  coordinate  system.  The  terms  as  and  Gl  represent  the  "short"  and 
"long"  standard  deviation  formulas  when  the  axes  are  rotated  to  eliminate  correlation: 


as  = 


(2-11) 


aL  = 


2 


(2-12) 


c  =  ellipticity  in  the  rotated  coordinate  system  =  Os  /  Gl  . 


(2-13) 


Let  MRmpi  represent  the  modified  Rand  R-234  estimator  for  CEPmpi  ,  where: 

MRmpi  =  .563  Ol  +  .614  Os  (if  c  >  .25)  (2-14) 

Next,  let  V  denote  the  bias  expressed  as  a  multiple  of  MRmpi  ■ 

v=^x^+f  /MRjvffi  .  (2-15) 

The  modified  Rand  R-234  estimator  is  valid  for  c  >  .25  and  v  <  2.2,  the  region  over  which 
Pesapane  and  Irvine  regressed  the  Rand-234  tables  [Pesapane  and  Irvine  (1977),  3-5]. 
Letting  MRand  refer  to  the  modified  Rand  R-234  CEP  estimator, 

MRand=  [  MRmpi  (  1.0039  -  0.0528v  +  0.4786v^  -  0.0793v^  )  ].  (2-16) 

4.  The  Valstar  CEP  Estimator.  This  algebraic  estimator  is  similar  to  the  modified  Rand 
R-234  estimator,  but  has  no  exclusionary  boundary  conditions  )^e  MRand. 

The  values  of  as,  at,  and  c  in  the  equation  that  follows  are  computed  identically  to 
the  MRand  method,  previously  described  in  equations  2-1,  2-2,  and  2-4.  The  equation  for 
the  estimator  of  the  CEP  with  respect  to  the  mean  point  of  impact,  denoted  ValstarMPi ,  is 
[Ethridge  (1983),  12]: 

ValstarMPi  =  {  0.562  Ol  +  0.615  Os  if  0.369  <  c  <  1 , 

0.675ol+-^  ifO  <c< 0.369  }  .  (2-17) 

L2cjl 

The  Valstar  CEP  estimator,  henceforth  referred  to  simply  as  Valstar,  is: 

Valstar  =  (V alstarMPi  +  x  ^  +  y  ^  .  (2-18) 

5.  The  Grubbs-Patniak  Chi-Square  CEP  Estimator.  Grubbs  presented  a  CEP  estimator 
based  on  the  assumption  that  the  x  and  y  components  of  the  radial  miss  distances  follow  a 
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normal  distribution.  If  this  assumption  holds,  then  the  sum  of  the  squares  of  these  r, 
components  follows  a  chi-square  distribution  [Grubbs  (1963),  53],  The  computation  of 
the  degrees  of  freedom  of  this  chi-square  distribution,  denoted  as  d,  requires  the 
computation  of  variables  defined  as  m  and  v.  Grubbs  is  used  to  denote  this  CEP  estimator 
in  this  study: 

m=  +  Sx^  +  Sy^  (2-19) 

V  =  2(Sx'^  +  2p^Sx^  Sy^  +  Sy‘*)+4  (x^  Sx^+  2xyp  Sx  Sy  +  y^Sy^)  (2-20) 

d  =  2m^v‘^  (2-21) 


Grubbs  = 


(2-22) 


6.  The  Rayleigh  Distribution  CEP  Estimator.  This  simple  algebraic  estimator  is  based 
on  the  assumption  that  the  radial  miss  distances  follow  a  Rayleigh  distribution  [Ethridge 
(1983),  10-11],  In  this  study,  we  refer  to  this  estimator  simply  as  Rayleigh.  The  formula 
for  Rayleigh  is: 


Rayleigh  =  . 93 94  f  (2-23  ) 

where  f  =  the  mean  of  the  radial  miss  distances. 

7.  The  Direct  Numerical  Integration  CEP  Estimator:  This  estimator  approximates  CEP 
using  the  following  double  integral  formula  for  p(r): 


r 

P(r)=  J  I 


r  2n  SxSyyj  l-p^ 


■/ 

\ 

1 

/'  — > 

x-x 

-2p 

(x-x)(y->0' 

2| 

\ 

\[  Sx  J 

S^Sy  j 

[Sy| 

dxdy 


(2-24) 


17 


This  integral  describes  the  volume  under  the  surface  formed  by  a  bivariate  normal 
distribution,  centered  at  (x  ,y ),  with  correlation  p  and  standard  deviations  Sx  and  Sy  with 
respect  to  the  crossrange  and  downrange  axes.  To  find  the  numerical  estimate  for  the 
CEP,  we  want  p(r)  to  equal  0.5.  The  corresponding  value  for  r,  approximated  using  a 
numerical  root  finding  method,  is  the  direct  numerical  integration  CEP  estimator.  It  is 
denoted  by  the  word  Numerical  in  this  thesis. 

8.  Tongue's  Modified  CBN  CEP  Estimator:  This  estimator  is  presented  in  Tongue  [1993, 
pages  6-1  through  6-6].  Using  regression  analysis.  Tongue  developed  a  formula  for 
predicting  the  relative  error  value  when  the  CEP  is  estimated  using  the  correlated  bivariate 
normal  (CBN)  numerical  integration  CEP  estimator  (or  any  numerical  integration  CEP 
estimator,  since  different  numerical  integration  methods  produce  nearly  identical  results). 
Tongue's  CBN  relative  error  estimator  is  a  function  of  the  sample  size,  scaled  sample  bias, 
and  sample  ellipticity  (where  sample  ellipticity  is  defined  by  equation  A- 10  of  this  thesis). 

The  numerical  CBN  CEP  estimator  is  based  on  the  equivalent  polar  coordinate 
form  of  the  double  integral  formula  for  p(r)  given  in  equation  2-24  [Elder  (1986),  3-6]; 


P(r)= - ^=T  f.=  < ‘'’"‘""•drde, 

2  7rS,SyVl-p'  0  0 


where: 


A  = 


sin^  0  2  p  sin0  cos0  cos^  0 

-  + - :r- 


2(1 -P^)  L  Sx^ 


(2-25) 


(2-26) 


B  = 


L 


xsin0  px  cos0  +  pysin0  ^  ycos0 


SxSy 


(2-27) 
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Next,  we  compute  Tongue's  relative  error  estimator 

Re  =  0.171833  -  0.009784«  -  0.037707  biaSs  -  0.  150628e% 

+  0.002045(k)(  btass)  +  0.007488(n)(  ellip)  +  0.019014(*fa5,)(  elltp) 

+  0.1 16385(e///»(p)  -  0.006714(e%)(p)(n)  (2-30) 

Tongue's  Modified  CEP  estimator,  denoted  as  TMCBN,  is  fhen  computed  by: 
JMCBN  =  CBN  {\-Re).  (2-3 1) 

We  close  this  chapter  by  presenting  an  example  which  demonstrates  that  these 
eight  estimators  can  (and  usually  do)  produce  differing  CEP  estimates  from  the  same  set 
of  sample  data.  Consider  the  following  sample  of  six  impact  coordinates: 


165  140 

37  -60 

Sample  =  217  -207 
81  -223 
113  119 

_257  -15^ 


The  resulting  sample  statistics  are: 


X  = 

145.0 

p  =  -0.42 

y  = 

-150.67 

f  =214.71 

Sx  “ 

83.56 

bias  =  209. 11  (2.881a) 

%  “ 
a  = 

59.60 

72.58 

II 

o^ 

o 
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The  eight  CEP  estimates  for  this  data  are: 

Smed=  226.82  Grubbs  =-  214.11 

Ethridge  =  194.52  Rayleigh  =  201.70 

MRand=  218.30  Numerical  =  216.92 

Valstar=  224.35  TMCBN=  220.20 

Summarizing  our  first  two  chapters,  we  have  discussed  basic  CEP  analysis 
concepts  and  described  each  estimator  used  in  our  thesis  experiment.  In  our  remaining 
chapters,  we  shift  the  focus  from  these  preliminary  topics  to  our  simulation  experiment 
design  and  the  analysis  of  its  results. 
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m.  DESCRIPTION  OF  THE  SIMULATION  EXPERIMENT 
We  designed  our  experiment  to  compare  the  eight  CEP  estimators  presented  in 
Chapter  2.  We  begin  this  chapter  by  examining  relationships  that  exist  between  our 
experimental  factors.  We  next  discuss  the  specific  factor  levels  used  to  form  our  design 
points  and  show  how  we  constructed  the  sample  analysis  sets,  which  are  based  entirely  on 
sample  statistics.  The  chapter  concludes  with  a  discussion  of  the  measures  of 
effectiveness  that  we  used  to  assess  the  relative  performance  of  the  eight  CEP  estimators 
considered. 

3.1  Relationships  Between  the  Experimental  Factors 

We  assumed  in  our  experiment  that  impacts  follow  a  bivariate  normal  distribution. 
In  selecting  the  levels  for  our  factors,  we  strived  to  meet  two  goals.  First,  to  provide  a 
comprehensive  study,  we  wanted  the  design  points  to  adequately  represented  a  wide  range 
of  bivariate  normal  distributions.  Second,  we  wanted  to  avoid  using  redundant  design 
points.  Several  relationships  that  exist  between  the  factors  used  in  our  experiment  enabled 
us  to  meet  these  two  goals  in  our  design. 

Consider  any  bivariate  normal  distribution  which  describes  the  points  of  impact  of 
incoming  projectiles.  Suppose  this  distribution  is  centered  at  (|Xx,|ty)  in  the  first  quadrant 
of  the  Cartesian  coordinate  system  representing  the  downrange  and  crossrange  miss 
distances,  with  the  angle  of  rotation  above  the  positive  X-axis  represented  by  0.  Next, 
consider  a  bivariate  normal  distribution  in  the  third  quadrant  with  identical  shape 
parameters,  centered  at  (-p,x ,  -fty  ),  and  an  identical  angle  of  rotation  0  formed  below  the 
negative  X-axis.  Because  of  the  symmetry  depicted  in  Figure  3.1,  the  radius  of  a  circle 


21 


containing  half  of  the  points  from  each  distribution  must  be  the  same.  Hence,  the  two 
distributions  will  have  identical  CEP  values.  This  same  relationship  holds  for  the  second 
and  fourth  quadrants. 


Y 


Figure  3.1  Identically  Shaped  Bivariate  Normal  Distributions  With  Equal  CEP  Values 

Consider  again  any  bivariate  normal  distribution  which  describes  the  points  of 
impact  of  incoming  projectiles,  centered  at  (px  ,Py)  in  the  first  quadrant  with  the  angle  of 
rotation  0.  Let  k  be  the  correlation  of  this  bivariate  normal  distribution.  Now  consider  a 
bivariate  normal  distribution  in  the  second  quadrant,  centered  at  (-px,  Py),  rotated  0  above 
the  negative  X-axis,  with  identical  shape  parameters  except  for  correlation,  which  is  -k. 
Because  of  the  shape  and  bias  of  the  2  distributions,  the  radius  of  a  circle  containing  half 
of  the  points  from  each  distribution  must  be  the  same;  therefore,  both  have  an  identical 
CEP  value.  An  example  of  this  relationship  is  displayed  in  Figure  3.2: 
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p  =  -k 


Figure  3.2  An  Example  of  Bivariate  Normal  Distributions  in  Quadrants  I  and  II  With 

Equal  CEP  Values 

Note  that  we  display  the  axes  rotated  at  an  angle  of  45°  in  Figure  3.2  and  other 
figures  in  this  paper.  This  rotation  is  due  to  a  software  limitation,  and  is  not  intended  to 
make  any  point.  Microsoft  Word  6.0  was  used  to  process  this  document,  and  this 
software  only  allows  the  construction  of  ellipses  aligned  completely  parallel  or  vertical  to 
the  horizontal. 

From  these  illustrations,  we  conclude  that  for  any  bivariate  normal  shape  in  the 
second,  third,  or  fourth  quadrant,  there  exists  a  bivariate  normal  distribution  in  the  first 
quadrant  with  an  identical  CEP  value.  Thus,  using  mean  points  of  impact  entirely  from 
the  first  quadrant,  we  can  adequately  represent  all  possible  bivariate  normal  distributions. 
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provided  we  use  both  positive  and  negative  correlation  levels. 

We  now  present  special  symmetries  which  exist  along  the  axes.  These  symmetries 
allowed  us  to  eliminate  certain  redundant  design  points  in  our  experimental  design.  An 
example  of  each  case  is  presented  in  Figure  3.3. 

Case  I:  Consider  the  special  case  of  a  bivariate  normal  distribution  with  correlation  k 
which  is  centered  upon  either  the  X  or  the  Y  axis.  Due  to  their  symmetrical  shapes,  a 
second  bivariate  normal  distribution,  centered  at  the  same  point  along  the  axis,  identical  in 
every  way  except  for  the  a  correlation  value  of  -k,  must  have  the  same  CEP  as  the  original 
distribution.  Thus,  for  the  special  case  of  bivariate  shapes  centered  along  one  of  the  axes, 
we  can  adequately  represent  all  possible  bivariate  normal  shapes  using  only  positive 
correlation  values. 

Case  II:  Any  bivariate  normal  shape  along  the  negative  X-axis  centered  at  (-{j,x ,  0)  with 
correlation  k  must  have  the  same  CEP  value  as  an  identically  shaped  bivariate  normal 
distribution  centered  at  (px  ,  0)  with  correlation  k  or  -k.  This  relationship  also  holds  true 
for  the  Y-axis.  We  can  therefore  adequately  represent  all  possible  bivariate  shapes  along 
the  X-axis  or  Y-axis  using  only  bivariate  normal  distributions  centered  along  the 
nonnegative  portion  of  the  axis. 

Case  III:  Consider  any  bivariate  normal  shape  centered  along  the  positive  Y-axis  at  (0,  p), 
with  crossrange  and  downrange  standard  deviation  values  of  a  and  b  respectively.  Now 
consider  a  second  bivariate  normal  shape  centered  along  the  X-axis  at  (p  ,  0)  with  an 
identical  correlation,  an  identical  bias  value,  and  crossrange  and  downrange  standard 
deviation  values  of  b  and  a  respectively.  Because  of  their  location  on  the  X  and  Y  axes. 
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identical  bias,  and  a  shape  which  is  identical  except  that  second  distribution  is  rotated 
ninety  degrees,  the  radius  of  a  circle  containing  half  of  the  points  from  each  distribution 
must  be  the  same.  Hence,  these  two  distributions  must  have  an  identical  CEP.  It  follows 
from  this  relationship  that  as  long  as  the  Oy  /a^  levels  are  reciprocals  (  such  as  2  and  1/2) 
we  can  use  design  points  centered  along  the  positive  X-axis  to  represent  all  of  the 
bivariate  normal  shapes  centered  along  either  the  X  or  Y  axis. 


Case  III:  Y 

The  two 
distributions 
centered  at 
points  A  and  B 
must  have  identical 
CEP  values. 


n 


Case  I: 

Both  distributions 
centered  at  point  C 
must  have  identical 
CEP  values. 


Case  II: 

The  two  distributions 
centered  at  point  C 
and  the  distribution 
centered  at  point  A 
must  have  identical 
CEP  values. 

IV 

CEP 


m 


Figure  3.3  Bivariate  Normal  Shapes  With  Equal  CEP  Values 


We  conclude  this  section  on  experiment  factor  relationships  by  exploring  the 
relationship  between  bias  and  the  CEP/CEPmpi  ratio.  If  this  ratio  is  very  large,  it  indicates 
that  there  is  a  "tight”  cluster  of  impacts  that  are  relatively  far  away  from  the  intended 
target.  All  other  things  being  equal,  one  logical  way  to  increase  the  probability  of  hitting 
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close  to  the  target  if  this  case  occurs  is  to  intentionally  aim  off  of  the  target  in  such  a  way 
that  the  tight  cluster  is  centered  on  target.  Thus,  the  statistical  parameters  of  the  tight 
cluster,  such  as  CEPmpi  ,  are  probably  more  significant  than  the  parameter  of  CEP  for  this 
circumstance. 

Consider  the  examples  in  Figure  3.4.  In  each  case  the  shape  of  the  bivariate 
normal  surface,  and  thus  the  CEPmpi  ,  is  identical.  The  only  difference  in  the  distributions 
is  the  center  point  and  bias  values.  The  example  illustrates  the  fact  that  for  any  bivariate 
normal  shape,  as  the  bias  increases,  so  does  the  CEP/CEPmpi  ratio.  It  follows  that  large 
CEP/CEPmpi  ratios  occur  only  when  the  bias  is  very  near  the  CEP  value. 


CEPO/CEPmpi  =  1;  CEPI/CEPmpi  =  6;  CEP2/CEPmpi  =  13;  CEP3/CEPmpi  =  20 
Figure  3.4  CEP/CEPmpi  Increases  as  Bias  Increases 

The  CEP/CEPmpi  ratios  for  the  experimental  design  points,  which  are  listed  in 
Appendix  C,  range  from  1.000  to  4.277.  The  largest  bias  level  used  in  the  experiment  is 
0.999  CEP;  to  produce  larger  CEP/CEPmpi  ratios,  larger  bias  levels  would  be  necessary. 
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3.2  The  Simulation  Experiment  Factors 


In  this  section,  we  present  each  of  our  five  experimental  factors  and  the  levels  used 
in  our  experimental  design  for  each  factor. 

1.  Correlation'.  In  earlier  studies,  Tongue(1993)  and  Puhek(1992)  used  only  bivariate 
normal  shapes  centered  along  the  positive  X-axis.  In  our  design,  all  of  our  bivariate 
normal  distributions  are  centered  within  the  first  quadrant,  but  not  all  along  the  X-axis. 
For  the  design  points  not  along  the  X-axis,  both  positive  and  negative  correlation  values 
are  necessary  to  adequately  represent  equivalent  bivariate  normal  distributions  in  all  of  the 
other  three  quadrants  for  the  reasons  described  in  Section  3.1.  Using  both  positive  and 
negative  correlation  levels  is  an  important  distinction  between  our  experimental  setup  and 
the  earlier  studies;  Tongue  and  Puhek  used  only  positive  correlation  values,  while 
Elder(1986)  did  not  use  correlation  as  a  factor  at  all. 

Correlation  values  can  range  from  -1.0  to  1.0.  For  our  experiment,  the  five 
uniformly  spaced  correlation  levels  of  -0.8,  -0.4,  0,  0.4,  and  0.8  were  selected.  Due  to  the 
symmetry  described  in  Section  3.1,  only  positive  correlation  values  were  used  for  design 
points  centered  on  the  X-axis. 

2.  The  Ratio  of  Oy/Ox  '.  Elder  (1986),  Puhek  (1992),  and  Tongue  (1993)  used  ellipticity 
as  a  factor.  In  this  study,  we  instead  use  the  ratio  of  downrange  standard  deviation  to 
crossrange  standard  deviation  (Oy  /c* ).  The  reason  for  this  choice  is  demonstrated  in  the 
following  example: 

Consider  the  following  two  impact  populations  below.  Both  have  an  identical 
CEP  of  100  and  are  identical  in  every  way  (including  ellipticity)  except  for  their  cjx  ,ay , 
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and  Oy  /Ox  values: 


P 

Px 

Py 

bias 

Ox 

Oy 

Oy /Ox 

ellipticity 

#1 

-0.4 

85 

0 

85 

108.9 

21.8 

0.2 

0.2 

#2 

-0.4 

85 

0 

85 

13.2 

65.9 

5.0 

0.2 

Figure  3.5  Contour  Lines  of  Example  Impact  Populations  #1  and  #2 

Observe,  as  suggested  in  Figure  3.5,  that  the  contours  of  the  bivariate  normal 
distribution  in  population  #1  will  enclose  much  larger  areas  than  the  corresponding 
contours  of  the  bivariate  normal  distribution  in  population  #2  due  to  the  larger  crossrange 
and  downrange  variances  in  population  #1.  This  example  shows  that  two  bivariate  normal 
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distributions  which  have  an  identical  mean  point  of  impact  and  ellipticity  can  require 
significantly  different  values  for  their  larger  and  smaller  standard  deviation  values  in  order 
to  achieve  an  equal  CEP.  By  using  Oy  /Cx  as  a  factor  instead  of  ellipticity,  and  by  using 
reciprocal  levels  for  Oy  /ax  (such  as  the  values  of  five  and  0.2  used  in  the  example),  we 
force  the  inclusion  of  both  such  bivariate  normal  shapes. 

The  five  levels  for  Oy/Cx  used  in  this  study  are  0.2,  0.6,1 ,  1.667,  and  5.  The  levels 
of  0.2,  0.6,  and  1  are  the  low,  middle,  and  high  values  from  the  five  ellipticity  levels  used 
in  Tongue's  1993  study.  The  levels  of  5  and  1.667  are  included  as  the  reciprocals  of  0.2 
and  0.6  respectively. 

3.  Bias:  In  order  to  maintain  generality,  we  use  the  following  scaling  factor  to  normalize 
our  bias  level  values: 


Tongue(1993)  and  Puhek(1992)  considered  bias  levels  range  between  zero  and  2a 
inclusively.  In  this  study,  we  extend  the  tested  bias  range  to  include  4a  by  using  the  bias 
levels  0,  0.5a,  1.0a,  2.0a,  and  4.0a. 

4.  Sample  Size:  Small  sample  sizes  are  of  particular  interest  in  this  study.  Tongue 
concluded  in  his  study  that  for  sample  sizes  greater  than  ten,  the  CBN  CEP  estimator  (or 
any  other  numerical  integration  CEP  estimator)  returned  the  least  biased  CEP  estimate  and 
recommended  that  future  studies  focus  on  sample  sizes  of  ten  or  less  [Tongue(1993),  7-7]. 
Based  on  this  recommendation,  we  chose  three  sample  size  levels  that  were  less  than  ten: 
three,  six,  and  nine.  We  also  included  a  sample  size  which  was  greater  than  ten,  fifteen,  to 
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test  Tongue's  assertion  with  our  design  point  results.  Note  that  Tongue's  assertion  was 
based  on  results  where  the  generating  population  was  known.  Using  a  sample  size  level  of 
fifteen  also  enabled  us  to  analyze  sample  analysis  sets,  based  strictly  on  sample  statistics, 
with  a  sample  size  which  was  greater  than  ten. 

5.  Rotation  Angle  9  \  The  uniformly  spaced  levels  of  0® ,  30®,  and  60®  were  selected  for 
our  final  experimental  factor,  the  angle  of  rotation  0  above  the  positive  X-axis.  As 
previously  stated,  (px,  Py)  for  all  of  our  design  points  lies  in  the  first  quadrant.  Thus,  our 
design  points  are  centered  according  to  their  bias  levels  either  along  the  X-axis  or  along 
imaginary  lines  rotated  30®  and  60®  above  the  x-axis. 

A  complete  list  of  the  275  design  points  used  for  each  sample  size  is  provided  in 
Appendix  C.  The  experimental  factors  and  the  levels  used  for  each  factor  are  summarized 
in  the  table  below: 


Table  3.1  Combinations  of  These  Values  are  Used  to  Form  the  Design  Points 


Sample  Size 

3, 

6, 

9, 

15 

Bias 

0, 

0.5c, 

2c, 

4a 

Correlation  (only  nonnegative  values 
used  along  the  X  axis) 

-0.8, 

-0.4, 

0, 

0.4, 

0.8 

av/cjx 

0.2, 

0.6, 

1, 

1.667, 

5 

Rotation  angle  (degrees)  above  the 
X-axis: 

0®, 

30®, 

60® 

Once  again,  it  is  emphasized  that  this  was  the  first  CEP  estimator  comparison 
study  which  used  replications  at  each  of  the  design  points.  Only  one  simulation  run  was 
accomplished  at  each  design  point  in  the  earlier  studies;  in  our  experiment,  ten  simulation 
runs  were  performed  at  each  design  point.  These  multiple  runs  allowed  us  to  compute 
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sample  statistics  at  each  design  point.  In  addition,  we  could  apply  the  measures  of 
effectiveness  discussed  in  Section  3.7  at  each  design  point. 

3.3  The  Design  Point  Templates: 

For  ease  in  comparing  the  eight  CEP  estimators,  especially  with  regards  to  the 
sample  analysis  set  results,  we  created  a  design  wherein  each  population  had  an  identical 
CEP  of  100.  Thus,  in  our  experiment,  the  expected  CEP  value  for  each  sample,  regardless 
of  design  point,  was  100. 

In  this  section,  we  explain  how  we  created  MathCAD  (1995)  templates  (programs) 
to  find  parameter  values  such  that  each  design  point  would  have  a  CEP  of  100. 

As  stated  in  Chapter  1,  if  the  population  parameters  of  the  underlying  bivariate 
normal  distribution  are  known,  then  CEP  is  best  approximated  using  numerical 
integration.  To  accomplish  this  calculation  using  direct  numerical  integration  requires  the 
following  population  parameters: 

1 .  The  crossrange  and  downrange  means. 

2.  The  crossrange  and  downrange  standard  deviations. 

3.  The  correlation  between  the  corresponding  crossrange  and  downrange  values. 

For  each  design  point,  however,  the  following  information  is  known: 

1 .  The  correlation  between  the  corresponding  crossrange  and  downrange  values. 

2 .  ^ • 

3.  Bias. 

4.  The  angle  of  rotation  0  above  the  positive  X-axis. 

Correlation  is  a  factor.  Therefore,  for  any  design  point  po ,  we  have  a  correlation 
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value  directly  assigned.  If  we  can  define  i^y ,  Oy ,  and  Ox  in  terms  of  |j,x  for  po ,  we  will 
have  only  one  unknown  (|j,x)  which  we  can  numerically  solve  for. 

First,  we  use  rotation  angle  0  to  define  Hy  in  terms  of  |ix  : 
tan  9  =  Hy/|ax  ^  |iy  =  jixtan  0.  (3-2) 

We  can  similarly  state  Ox  in  terms  of  Oy  using  Oy/Ox.  This  ratio  is  set  at  one  of  five 
levels  for  each  design  point.  Let  Oy /Ox  equal  ki  for  po .  We  can  algebraically  restate  the 
equation  as; 

Oy  =  ki  Ox  (3-3) 

The  bias  of  po ,  is  assigned  one  of  five  levels,  each  of  which  can  be  represented  as  a 
constant  k2  multiplied  by  o.  By  definition,  we  can  also  represent  the  bias  of po  in  terms  of 
Px  and  Py  .  Below,  we  set  these  two  equivalent  expressions  for  the  bias  of  po  equal  to 
each  other,  then  substitute  in  the  values  firom  equations  3-2  and  3-3: 
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Substituting  for  in  equation  3-3, 


Oy  = 


kitix 


sec0 


(3-5) 


Using  equations  3-3  and  3-5,  we  constructed  MathCAD  (1995)  templates  to  find 
parameters  such  that  each  design  point  would  have  a  CEP  =  100  within  a  tolerance  of 
0.01.  An  example  of  one  of  these  templates  is  provided  in  Appendix  B.  Note  the 
accuracy  of  the  CEP  values  listed  in  the  template  output  for  this  example. 

3.4  The  Sample  Analysis  Sets 

Recall  that  for  each  sample  size  the  term  "sample  analysis  set"  refers  to  a  set  of 
generated  sample  data  whose  sample  bias,  sample  correlation,  and  sample  crossrange  to 
sample  downrange  standard  deviation  ratio  fall  into  given  specific  ranges. 


For  each  of  these  three  sample  statistics,  we  used  ranges  for  the  sample  analysis 
sets  which  were  roughly  centered  around  the  design  factor  levels.  We  first  display  the 
sample  bias  ranges  used: 


Population  Bias 
Levels 

0  0.5a 

1.0a 

2.0a 

4.0a 

Sample  Bias 
Ranges 

[0,  0.75  a] 

(0.75  a,  1.255] 

(1.255,2.755] 

(>2.755) 

Although  it  would  appear  that  the  first  sample  bias  range  of  [  0,  0.75  o]  would 
apply  to  more  outcomes  than  the  other  three  ranges  (since  the  expected  value  of  the 
sample  bias  for  design  points  where  the  bias  equals  zero  or  0.5a  falls  into  this  range),  high 
variance  within  the  small  sample  sizes  used  resulted  in  a  relatively  uniform  allocation  in 
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each  of  the  four  sample  bias  categories. 

The  sample  correlation  ranges  were  centered  directly  around  the  population 
correlation  levels  as  follows: 


Population  Correlation 
Levels 

-0.8 

-0.4 

0 

0.4 

0.8 

Sample  Correlation 
Ranges 

[-1.0,-0.6) 

[-0.6,-0.2) 

[-0.2,0.2] 

Finally,  the  sample  analysis  set  Sx  /Sy  ranges  were  centered  around  the  population 
Ox  /Oy  levels,  except  for  the  open-ended  range  ( >  2.5  ): 


Population  /Oy 
Levels 

0.2 

0.6 

1.0 

1.667 

5.0 

Sample  Sx/Sy 
Ranges 

(0,  0.4) 

[0.4,  0.8) 

[0.8,  1.25] 

(1.25,  2.5] 

(>2.5) 

All  of  the  2,750  samples  generated  for  each  sample  size  were  sorted  into  sample 
analysis  sets  for  the  comparison  of  the  eight  CEP  estimators  based  strictly  on  sample 
statistics.  The  combinations  used  to  form  the  sample  analysis  sets  for  each  sample  size  are 
summarized  below  in  Table  3.2;  a  complete  list  of  the  sample  analysis  sets  for  each 
sample  size  is  presented  in  Appendix  D. 


Table  3.2  Combinations  Used  to  Form  the  Sample  Analysis  Sets  for  Each  Sample  Size 


Sample  Bias  Range 

[0,0.75a],  (0.755,1.25a],  (1.255,2.755],  (>2.755) 

P  Range 

[-1.0,  -0.6),  [-0.6,  -0.2),  [-0.2,  0.2],  (0.2,  0.6  ],  (0.6,  1.0] 

Sx/Sy  Range 

(>  2.5),  (1.25,2.51,  [0.8,1.25],  [0.4, 0.8),  (<0.4) 

3.5  The  Sample  Generator  Proaram 

In  this  section,  we  describe  the  program  used  to  generate  the  simulated  samples. 
In  our  program  design,  we  took  advantage  of  the  fact  that  we  can  form  two  new  axes  (X' 
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and  7')  and  "rotate"  them  such  that  no  correlation  is  present  in  the  new  coordinate  system. 
As  described  by  Tongue  [(1993),  pages  3-12  and  3-13],  the  angle  of  rotation  6  is: 


0 


— arctan 
2 


2  P  CTy 


The  values  for  px',  Py',  ,  and  Cy  in  this  new  rotated  axis  system  are: 
Px'  =  Px  COS0  +  Py  sin0 
Py'  =  -px  sin0  +  Py  COS0 
If  Ox  >  Oy : 


If  Ox  <  Oy  : 


2 


+  )/(^x^+^  y^j  +  4paxay 


(3-6) 

(3-7) 

(3-8) 

(3-9) 

(3-10) 

(3-11) 

(3-12) 


Our  sample  generation  program,  listed  in  Appendix  E,  is  written  in  the  MOD  SIM 
programming  language  [CACI  (1995)].  This  program  first  transforms  the  input  Px ,  Py , 
Ox ,  and  Oy  values  into  px' ,  Py  ,  Ox ',  and  Oy'.  Next,  for  the  input  sample  size,  random  x'  and 
y'  values  are  selected  from  the  bivariate  normal  distribution. 
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To  apply  our  CEP  estimators  to  the  simulated  samples  created,  the  (x',  y')  values 
generated  were  next  transformed  back  into  their  corresponding  (x,  y)  values  in  the  original 
X  and  Y  coordinate  system  using  these  two  equations: 

X  =  x'cosG  -  y'sinG  (3-13) 

y  =  x'sinG  +  y'cosG  (3-14) 

The  program  requires  the  sample  size  to  be  manually  input.  Each  line  of  an  input 
file  for  the  program  consists  of  a  given  design  point's  crossrange  mean ,  downrange  mean, 
crossrange  standard  deviation,  and  downrange  standard  deviation  values,  plus  the  number 
of  runs  desired  for  the  given  design  point.  The  output  for  each  run  consists  of  the  sample 
size,  sample  correlation,  sample  crossrange  mean  ,  sample  downrange  mean,  sample 
crossrange  standard  deviation,  sample  downrange  standard  deviation,  sample  median, 
sample  mean  radial  error,  and  the  Ethridge  CEP  estimate. 

To  verify  that  the  program  produces  reliable  output,  a  file  containing  20  varied 
design  points,  each  with  a  population  CEP  of  100,  was  created.  For  this  test  run,  the 
sample  size  was  set  to  1000.  The  output  from  this  file  showed  that  the  generated  points 
accurately  reflected  the  input  bivariate  normal  shape  characteristics.  The  input  and  output 
files  are  presented  in  Tables  3.3  and  3.4.  One  can  observe  the  accuracy  of  the  program  by 
comparing  the  Px,  Py ,  Ox,  ay ,  and  p  values  of  the  input  file  with  their  corresponding  x,  y, 
Sx,  Sy ,  and  p  sample  statistic  values  in  output  file.  In  Table  3.5,  the  relative  error  between 
each  of  these  five  parameters  and  their  corresponding  sample  statistics  is  presented  for 
comparison. 


36 


Table  3.3  The  Input  File  Used  to  Verify  the  Sample  Generator  Program 


Repetitions 

Py 

CTx 

Ov 

1 

73.6 

42.5 

98.5 

19.7 

-0.8 

1 

35 

60.6 

103.2 

20.6 

-0.8 

1 

60.6 

35 

85.7 

51.4 

6 

bo 

1 

60.6 

35 

20.6 

103.2 

-0.8 

1 

42.5 

73.6 

19.7 

98.5 

-0.8 

1 

35 

60.6 

23.2 

116 

-0.8 

1 

73.6 

42.5 

98.4 

19.7 

-0.4 

1 

60.6 

35 

115.3 

23.1 

-0.4 

1 

35 

60.6 

63.1 

63.1 

-0.4 

1 

60.6 

35 

45.6 

75.9 

-0.4 

1 

60.6 

35 

101.2 

-0.4 

1 

42.5 

73.6 

19.7 

98.4 

-0.4 

1 

35 

60.6 

23 

115.1 

-0.4 

1 

73.6 

42.5 

103.4 

20.7 

0.4 

1 

60.6 

35 

71.8 

71.8 

0.4 

1 

35 

60.6 

55,4 

92.4 

0.4 

1 

35 

60.6 

111 

22.2 

0.8 

1 

73.6 

42.5 

71.8 

71.8 

0.8 

1 

60.6 

35 

58.1 

96.8 

0.8 

1 

42.5 

73.6 

21.4 

106.9 

0.8 

Table  3.4  The  Resulting  Output  for  the  Input  File  in  Table  3.3: 


Sample  Si2e 

X 

y 

Sx 

Sy 

P 

Smed 

..  I 

Ethridge 

1000 

75.325 

42.295 

98.565 

19.643 

-0.802 

99.704 

117.11 

103.65 

1000 

34.498 

60.515 

101.1 

20.193 

-0.794 

98.668 

113.43 

104.06 

1000 

59.041 

35.225 

83.738 

49.721 

-0.793 

97.794 

107.46 

96.107 

1000 

60.449 

36.033 

21.19 

-0.811 

99.471 

115.81 

105.81 

1000 

41.964 

78.586 

19.745 

-0.792 

99.655 

119.3 

104.58 

1000 

34.7 

63.228 

22.822 

115.07 

-0.792 

98.182 

117.93 

100.82 

1000 

75.101 

42.788 

98.271 

-0.393 

102.28 

115.89 

100.2 

1000 

59.621 

35.147 

115.24 

-0.36 

99.953 

116.34 

96.897 

1000 

35.595 

61.578 

62.365 

63.807 

-0.403 

99.809 

103.41 

90.807 

1000 

60.747 

33.893 

45.372 

74.934 

-0.397 

97.562 

102.26 

91.853 

1000 

60.526 

34.018 

19.94 

101.39 

-0.33 

100.23 

112.96 

102.6 

1000 

42.493 

70.016 

19.919 

96.985 

-0.418 

98.714 

113.7 

99.475 

1000 

35.524 

59.577 

22.593 

WIW 

-0.359 

97.781 

113.94 

95.348 

1000 

74.591 

42.183 

102.95 

0.4124 

99.626 

116.38 

97.135 

1000 

60.58 

35.625 

73.966 

71.307 

0.4239 

100.35 

109.14 

91.89 

1000 

34.881 

63.196 

54.846 

90.963 

0.3873 

97.364 

110.23 

90.924 

1000 

33.602 

60.234 

110.48 

22.43 

0.8078 

100.35 

117.27 

104.17 

1000 

72.226 

40.452 

71.417 

71.331 

0.7956 

96.699 

111.02 

90.079 

1000 

59.333 

33.028 

59.428 

97.898 

0.8166 

104 

114.26 

95.239 

1000 

42.688 

76.026 

22.07 

110.3 

0.8062 

98.492 

119.13 

95.821 
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Table  3.5  Relative  Error  (RE)  of  the  Corresponding  Sample  Statistics  For  the  Input 

Population  Parameters  in  Table  3.3 


mm 

X 

RE 

M-y 

y 

RE 

P 

p 

RE 

73.6 

75.325 

0.0234 

42.5 

42.295 

0.0048 

-0.8 

-0.802 

0.0027 

35 

34.498 

0.0143 

60.6 

-0.8 

-0.794 

0.0078 

60.6 

59.041 

0.0257 

35 

35.225 

0.0064 

-0.8 

-0.793 

0.0088 

60.6 

60.449 

0.0025 

35 

36.033 

0.0295 

-0.8 

-0.811 

0.0136 

42.5 

41.964 

0.0126 

73.6 

78.586 

0.0677 

-0.8 

-0.792 

0.0095 

35 

34.7 

0.0086 

60.6 

63.228 

0.0434 

-0.8 

-0.792 

0.0106 

73.6 

75.101 

42.5 

42.788 

0.0068 

-0.4 

-0.393 

0.0188 

60.6 

59.621 

0.0162 

35 

35.147 

0.0042 

-0.4 

-0.36 

0.1005 

35 

35.595 

0.017 

60.6 

61.578 

0.0161 

-0.4 

-0.403 

0.0075 

60.6 

60.747 

0.0024 

35 

33.893 

0.0316 

-0.4 

-0.397 

0.0068 

60.6 

60.526 

0.0012 

35 

34.018 

0.0281 

-0.4 

-0.33 

0.1758 

42.5 

42.493 

0.0002 

73.6 

70.016 

0.0487 

-0.4 

-0.418 

0.0443 

35 

35.524 

0.015 

60.6 

59.577 

0.0169 

-0.4 

-0.359 

0.1023 

73.6 

74.591 

0.0135 

42.5 

42.183 

0.0075 

0.4 

0.4124 

0.031 

60.6 

60.58 

0.0003 

35 

35.625 

0.0178 

0.4 

0.4239 

0.0598 

35 

34.881 

0.0034 

60.6 

63.196 

0.0428 

0.4 

0.3873 

0.0319 

35 

33.602 

0.0399 

60.6 

60.234 

0.006 

0.8 

0.8078 

0.0097 

73.6 

72.226 

0.0187 

42.5 

0.8 

0.7956 

0.0055 

60.6 

59.333 

0.0209 

35 

33.028 

0.0563 

0.8 

0.8166 

0.0208 

42.5 

42.688 

0.0044 

73.6 

76.026 

0.033 

0.8 

0.8062 

0.0077 

t7x 

Sx 

RE 

Gy 

RE 

Median 

Smed 

RE 

98.5 

98.565 

0.0007 

19.7 

19.643 

0.0029 

100 

99.704 

0.003 

103.2 

101.1 

0.0203 

20.6 

20.193 

0.0198 

100 

98.668 

0.0133 

85,7 

83.738 

0.0229 

51.4 

49.721 

0.0327 

100 

97.794 

0.0221 

20.6 

21.19 

0.0286 

103.2 

104.74 

0.0149 

100 

99.471 

0.0053 

19.7 

19.745 

0.0023 

98.5 

101.52 

0.0307 

100 

99.655 

0.0035 

23.2 

22.822 

0.0163 

116 

115.07 

0.008 

100 

98.182 

0.0182 

98.4 

98.271 

0.0013 

19.7 

19.575 

0.0063 

100 

102.28 

0.0228 

115.3 

115.24 

0.0005 

23.1 

22.842 

0.0112 

100 

99.953 

0.0005 

63.1 

62.365 

0.0116 

63.1 

63.807 

0.0112 

100 

99.809 

0.0019 

45.6 

45.372 

0.005 

75.9 

74.934 

0.0127 

100 

97.562 

0.0244 

20.2 

19.94 

0.0129 

101.2 

101.39 

0.0018 

100 

100.23 

0.0023 

19.7 

19.919 

0.0111 

98.4 

96.985 

0.0144 

100 

98.714 

0.0129 

23 

22.593 

0.0177 

115.1 

111.4 

0.0321 

100 

97.781 

0.0222 

103.4 

102.95 

0.0043 

20.7 

20.386 

0.0152 

100 

99.626 

0.0037 

71.8 

73.966 

0.0302 

71.8 

71.307 

0.0069 

100 

100.35 

0.0035 

55.4 

54.846 

0.01 

92,4 

90.963 

0.0156 

100 

97.364 

0,0264 

111 

110.48 

0.0047 

22.2 

22.43 

0.0104 

100 

100.35 

0.0035 

71.8 

71,417 

0,0053 

71.8 

71.331 

0.0065 

100 

96,699 

0.033 

58.1 

59.428 

0.0229 

96.8 

97.898 

0.0113 

100 

104 

0.04 

21.4 

22.07 

0.0313 

106.9 

110.3 

0.0318 

100 

98.492 

0.0151 
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3.6  Generating  the  CEP  Estimates  From  the  Sample  Generator  Output  Data 


Given  any  set  of  MODSIM  generator  program  output,  our  MathCAD  CEP 
estimator  template  listed  in  Appendix  F  computes  the  CEP  value  for  each  of  the  eight  CEP 
estimators  considered  in  our  experiment.  In  addition,  the  template  also  calculates  the 
measures  of  effectiveness  discussed  in  Section  3.7. 

The  generated  output  for  each  design  point,  where  the  imderlying  population 
parameters  were  known,  and  for  each  sample  analysis  set,  where  the  underlying  population 
parameters  were  unknown,  were  processed  through  this  template. 

The  run  time  for  the  MathCAD  CEP  estimator  template  with  a  100  MHz.  Pentium 
processor  equipped  P.C.  was  normally  less  than  two  minutes  per  design  point,  or  around 
five  minutes  for  a  thirty  element  sample  analysis  set. 

In  retrospect,  one  aspect  that  we  failed  to  address  in  our  MathCAD  CEP  estimator 
template  was  the  identification  of  design  points  which  were  invalid  for  MRcmd.  This 
estimator  is  only  valid  for  certain  cases  described  in  Section  2.3.  By  processing  every 
design  point  for  MRand,  it  was  difficult  to  fairly  evaluate  this  estimator. 

3.7  Measures  of  Effectiveness 

We  use  the  term  "measures  of  effectiveness",  or  MOEs,  to  refer  to  the  criteria  used 
to  distinguish  between  CEP  estimators.  Before  we  present  the  MOE  which  we  selected 
for  evaluating  our  experimental  results,  we  first  discuss  the  MOEs  used  in  the  earlier 
studies  of  Elder  (1986),  Puhek  (1992),  and  Tongue  (1993). 

Elder  and  Puhek  reached  their  conclusions  by  identifying  the  CEP  estimator  with 
the  least  relative  error  (RE)  for  the  majority  of  their  design  point  results,  where  RE  is 
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defined; 


RE  =  I  actual  CEP  -  estimated  CEP  1  /  actual  CEP.  (3-15) 

Unfortunately,  comparing  the  CEP  estimators  using  only  RE  ignores  the  important 
characteristic  of  variance. 

Consider  the  two  distributions  of  CEP  estimates  in  Figure  3.6  below.  Based 
strictly  on  minimizing  bias  or  relative  bias,  distribution  #1  would  be  “best".  However,  the 
high  variance  of  distribution  #1  versus  distribution  #2  illustrates  the  need  for  a  more 
effective  MOE  to  compare  these  two  CEP  estimate  distributions. 


Distribution#! 

\  Distribution  #2 

. 

Actual  CEP  CEP#1  CEP  #2 

Figure  3.6  Distribution  #1  has  Less  Bias  but  Higher  Variance  than  Distribution  #2 

Instead  of  determining  which  CEP  estimator  had  the  least  relative  error  for  the 
majority  of  his  design  point  results.  Tongue  considered  the  mean  relative  error  (MRE)  of 
each  CEP  estimator  in  his  1993  study.  In  addition.  Tongue  considered  two  other  MOEs. 
The  first  of  these  was  to  compare  which  estimator  had  the  least  variance  among  it's  MRE 
value;  we  use  VRE  hereafter  to  refer  to  this  MOE.  Tongue's  second  additional  MOE  was 
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the  mean  square  of  the  relative  error,  hereafter  denoted  as  MSRE,  which  is  defined  in 
either  of  the  following  manners: 

M5R£:= -yRE.'  =  MRE^  +  VRE  (3-16) 

"tr 

In  our  study,  we  computed  the  results  for  all  three  MOEs  from  Tongue's  study  for 
every  design  point  and  sample  analysis  set  result.  In  our  actual  analysis,  however,  we 
relied  on  the  MSRE  instead  of  MRE  or  VRE  for  the  following  two  reasons: 

1.  MSRE  combines  the  effectiveness  characteristics  of  both  MRE  and  VRE. 

2.  As  seen  in  Appendices  I  and  J,  the  estimator  with  the  best  MSRE  score  for  a  given  set 
of  conditions  was  also  virtually  always  best  for  either  MRE  or  VRE,  and  usually  both. 

To  demonstrate  how  an  MSRE  difference  between  estimators  relates  to  the 
relative  error  between  the  estimators,  we  offer  the  plot  in  Figure  3.7: 


Figure  3.7  Relationship  of  MSRE  to  MRE  and  VRE 
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Consider  a  CEP  estimator  which  always  yields  an  estimate  with  an  error  of  exactly  five 
percent.  In  terms  of  RE,  this  difference  would  equal  0.05.  In  terms  of  MSRE,  this 
difference  would  correspond  to  0.05  squared,  or  0.0025.  As  seen  in  Figure  3.7,  an  MSRE 
of  amount  X  equates  to  a  relative  error  of  -v/Xif  the  variance  is  zero.  Therefore,  if  the 
MSRE  is  known,  the  square  root  of  the  MSRE  provides  an  approximation  for  the  MRE. 
This  approximation  for  the  MRE  multiplied  by  the  actual  parameter  being  estimated  (in 
our  case  CEP)  yields  an  approximation  of  the  actual  error  one  could  expect  based  on  the 
known  MSRE.  We  hereafter  use  AE  to  refer  to  this  MSRE-based  approximation  of  error. 

We  use  the  MOEs  of  MSRE  and  AE  extensively  in  the  analysis  of  our 
experimental  results  in  the  remaining  two  chapters.  In  Chapter  4,  we  outline  these 
experimental  results,  while  in  Chapter  5  we  present  recommendations  based  on  these 
results. 
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IV.  THE  THESIS  EXPERIMENT  RESULTS 


In  this  chapter,  we  present  the  results  of  the  experiment  described  in  Chapter  3. 
We  followed  an  identical  experimental  process  for  both  the  design  points  and  sample 
analysis  sets  to  analyze  the  differences  and  similarities  in  their  results.  The  experimental 
protocol  consisted  of 

1.  Presenting  overall  observations  of  the  resulting  data. 

2.  Investigating  which  experimental  factors  appeared  to  be  most  significant. 

3.  Investigating  combinations  of  those  factors  determined  to  be  the  most  significant. 

In  addition  to  comparing  our  design  point  results  against  our  sample  analysis  set 
results,  we  also  compared  our  design  point  results  with  those  of  previous  studies. 

4.1  Convereence  Deviations  for  the  Numerical  and  TMCBN  CEP  Estimators 

In  our  MathCAD  CEP  estimator  template  described  in  Section  3.6,  we  designated 
a  precision  tolerance  of  0.01  for  the  calculations  of  all  eight  CEP  estimators.  The 
implication  of  this  tolerance  for  the  root  finding  algorithm  employed  by  Numerical  and 
TMCBN v/as  that  r  was  approximated  such  that  0.49  <  p(r)  <0.51.  We  discovered  design 
points  and  sample  analysis  sets,  however,  where  our  root  finding  algorithm  would  not 
converge  at  this  tolerance.  These  deviations  occurred  primarily  at  sample  size  three. 

These  deviant  design  points  and  sample  analysis  sets  are  listed  in  Table  4.1.  When 
the  root  finding  algorithm  of  Numerical  and  TMCBN  did  not  converge  using  a  tolerance 
of  0.01,  we  systematically  tried  larger  tolerances  of  0.1,  0.2,  0.5,  and  1.0.  Any  design 
point  or  sample  analysis  set  that  would  not  converge  at  a  tolerance  of  1.0  was  conceded  as 
non-convergent: 
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Table  4.1  Design  Points/Sample  Analysis  Sets  Where  a  Tolerance  of  Other  Than  0.01 

Was  Used  for  Numerical  and  TMCBN 


Sample 

Size 


6 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


3 


Design  Pt. 
Reference  # 


230 


9 


20 


24-25 


30 


33-34 


40 


55 


57 


69 


80 


91 


95-98 


111 


112 


114 


117 


120 


122-124 


131 


132 


142-143 


145 


147 


151 


152 


155 


159-160 


161 


162 


165-166 

170 

174 

175 
179 

181-182 
(continued) 


Tolerance 

Used 


0.1 


0.2 


0.1 


0.1 


0.1 


0.1 


0.5 


0.1 


0.1 


0.1 


0.1 


0.1 


0.1 


0.2 


0.1 


0.1 


0.1 


0.2 


0.1 


Sample 

Size 


6 


3 


3 


Sample  Analysis 
Set  Reference  # 


80 


Tolerance 

Used 


0.1 


-  Indicates  no 
convergence 
for  the  given 
design  point  or 
sample  analysis 
set. 


Sample 

Design  Pt. 

Tolerance 

Size 

Reference  # 

Used 

3 

183-187 

0.1 

3 

188 

0.2 

3 

189 

0.1 

3 

190 

0.2 

3 

191-201 

0.1 

3 

203-205 

0.1 

3 

210-214 

0.1 

3 

215 

0.2 

3 

216-220 

0.1 

3 

221 

0.2 

3 

222-223 

0.1 

3 

224 

0.5 

3 

225-226 

* 

3 

227 

0.2 

3 

228 

* 

3 

229 

0.1 

3 

230 

* 

3 

231 

0.1 

3 

232 

0.5 

3 

233-234 

0.1 

3 

235 

1.0 

3 

236-239 

0.1 

3 

240 

* 

3 

241-244 

0.1 

3 

245 

* 

3 

246-248 

0.1 

3 

249 

* 

3 

250-251 

0.1 

3 

252 

0.5 

3 

253-355 

0.1 

3 

256 

* 

3 

257-263 

0.1 

3 

264 

0.2 

3 

265-269 

0.1 

3 

270 

1.0 

3 

271 

* 

3 

272 

0.5 

3 

273 

0.1 

3 

274 

* 

3 

275 

0.1 

TOTALS  FOR  SAMPLE  SIZE  =  6 


Tolerance 

Used 

Design 

Points 

Sample  Analysis 
Sets 

0.1 

1 

1 

TOTALS  FOR  SAMPLE  SIZE  =  3 

Tolerance 

Used 

Design 

Points 

Sample  Analysis 
Sets 

0.1 

99 

8 

0.2 

14 

6 

0.5 

9 

0 

1.0 

2 

2 

* 

12 

16 

NOTE 

*  -  Indicates  no 
convergence 
for  the  given 
design  point  or 
sample  analysis 
set. 
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Note  from  Table  4.1  that  all  of  the  non-convergent  cases  occurred  exclusively 
when  the  sample  size  equaled  three.  These  non-convergent  cases  appear  related  to  the 
Gy/Ox  and  Sy/Sx  parameters,  based  on  the  patterns  displayed  in  Table  4.2: 

Table  4.2  Relationship  Between  Gy/Ox,  Sy/Sx,  and  Non-Convergent  Cases 


Sy /Sx 

Number  of  Non-Convergent 
Sample  Analysis  Sets 

WBBEBM 

1 

■lifisMl 

5 

[0.8,1.25] 

2 

6 

2 

Gy/Gx 

Number  of  Non- 
Convergent  Design  Points 

0.2 

5 

0.6 

0 

1 

1 

2 

5 

4 

Tongue  (1993)  had  similar  convergence  problems  with  his  numerical  CBN  CEP 
estimator  in  his  study.  He  determined  that  convergence  problems  occurred  when  the 
sample  statistics  from  a  highly  elliptical  population  reflected  a  predominately  circular 
distribution  or  vice-versa  [Tongue  (1993),  page  4-16],  As  seen  in  Table  4.2,  our  non- 
convergent  design  points  tend  to  originate  from  populations  of  extreme  ellipticity;  we 
believe  that  sample  statistics  reflecting  a  predominately  circular  distribution  were 
generated  from  these  design  points.  Our  non-convergent  sample  analysis  sets,  on  the 
other  hand,  tend  to  have  Sy  /Sx  values  that  indicate  a  predominately  circular  distribution. 
We  believe  that  these  non-convergent  sample  analysis  sets  contain  the  output  from  one  or 
more  of  the  non-convergent  design  points. 

Because  a  relatively  large  number  of  Numerical  and  TMCBN  estimates  were 
computed  using  a  tolerance  of  less  than  0.01  at  sample  size  three,  the  MOE  scores  of 
these  two  estimators  at  this  sample  size  were  distorted.  Given  more  time,  we  would  have 
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investigated  the  performance  of  Numerical  and  TMCBN  at  sample  size  three  for  only 
those  design  points  and  sample  analysis  sets  that  converged  at  a  tolerance  of  Q.Ol. 

4»2  The  Design  Point  Results 

Now  that  we  have  outlined  the  convergence  problems  for  the  two  numerical 
estimators  considered,  we  next  present  out  design  point  results. 

The  total  experimental  output  was  voluminous,  as  evidenced  by  the  design  point 
and  sample  analysis  set  MSRE  results  presented  in  Appendices  G  and  H.  Our  first  attempt 
at  interpreting  this  large  volume  of  data  was  to  determine  for  all  design  point  and  sample 
analysis  set  cases  which  estimator  had  the  most  "best  finishes"  (lowest  scores)  for  the 
MOEs  of  MRE,  VRE,  and  MSRE.  These  values  are  recorded  in  Appendices  I  and  J 
respectively.  While  these  numbers  give  an  indication  of  how  the  considered  CEP 
estimators  performed,  they  can  also  be  misleading. 

Suppose  that  estimators  A  and  B  are  being  compared  for  a  given  case  containing 
thirty  design  points.  Suppose  further  that  when  the  MSRE  values  are  compared,  estimator 
A  "wins"  twenty  times  while  estimator  B  "wins"  only  ten  times.  Estimator  A  appears  to 
be  the  better  estimator.  What  is  unknown  from  these  numbers,  however,  is  the  magnitude 
for  each  of  the  respective  "wins."  Suppose  that  when  A  wins  the  margin  between  A  and  B 
is  very  narrow,  while  when  B  wins  the  margin  is  very  great.  Clearly  a  more  effective  tool 
is  needed  to  compare  these  estimators  than  just  the  number  of  "wins." 

We  decided  for  most  cases  that  the  best  way  to  display  the  performance  of  the 
competing  estimators  was  to  construct  cumulative  distribution  function  (CDF)  plots 
relating  the  MSRE  results  of  the  competing  estimators  to  cumulative  probability. 
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When  comparing  CDFs,  a  distribution  A  "stochastically  dominates"  another 
distribution  B  for  a  given  set  of  continuous  random  numbers  X  if  the  following  condition 
is  met  [Clemen  (1996),  238]: 

Fa(x)  <  Fb(x)VxgX  (4-1) 

Stochastic  dominance  is  indicated  graphically  by  the  plot  of  the  CDF  of  A  always  lying 
equal  to  or  below  the  plot  of  the  CDF  of  B: 


Figure  4.1  Distribution  A  Stochastically  Dominates  Distribution  B 
In  our  situation,  good  performance  is  indicated  by  low  MSRE  scores.  Therefore,  an 
estimator  that  stochastically  dominates  another  can  be  considered  inferior. 

We  begin  our  presentation  of  the  design  point  results  by  displaying  histograms  for 
each  CEP  estimator  considered.  An  estimator  that  has  primarily  low  MSRE  values  shows 
large  bars  on  the  left  side  of  the  graph;  conversely,  large  bars  on  the  right  hand  side  of  the 
graph  indicate  that  the  given  estimator  had  a  large  number  of  design  points  with  a 
relatively  high  CEP  estimation  error.  Note  that  the  far  right  bar  refers  to  an  open-ended 
range  of  all  design  points  where  the  MSRE  was  >  0.25: 
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Figure  4.2  Overall  Histograms  Based  on  the  Design  Point  Results 
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Number  of  Design  Points  I  Number  of  Design  Points  I  Number  of  Design  points 
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Figure  4.2  Overall  Histograms  Based  on  the  Design  Point  Results  (continued) 
Considering  these  histograms,  no  single  estimator  stands  out  as  being  dominated 
by  the  other  estimators.  Each  estimator  except  MRand  shows  a  similar  distribution  of  bars 
descending  left  to  right.  These  histograms  do,  however,  indicate  weak  performance  from 
Stned  and  MKawi/ based  on  the  relatively  larger  number  of  design  points  which  had  MSRE 
values  in  excess  of  0.25.  The  histogram  for  MRand  is  not  surprising;  as  alluded  to  in 
Section  3.6,  we  calculated  the  MRand  estimate  for  all  design  points,  even  those  outside 
the  estimator's  validity  bounds. 
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We  next  present  the  average  MSRE  values,  the  corresponding  approximation  for 
the  MRE  based  on  these  MSRE  averages,  and  the  AE  for  each  estimator  below  (bear  in 
mind  that  the  actual  CEP  was  100); 


— ..^E^mator 
Statistic  - __ 

Smed 

Ethridge 

MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

1.6525 

0.0478 

0.0391 

0.0391 

0.0421 

0.0399 

Corresponding  MRE 
Approximation 

0.1995 

1.2855 

0.2187 

0.1977 

0.1976 

0.2051 

0.1997 

AE  (True  CEP  was  100) 

+23.758 

±19.951 

+128.55 

±21.873 

±19.769 

±19.764 

±20.507 

±19.969 

Table  4.3  Overall  Design  Point  Statistics 


MRand,  Smed,  and  Valstar  appear  relatively  weak  in  Table  4.3,  since  they  have  the 
highest  overall  error  statistics.  CDF  plots  based  on  the  overall  design  point  results  further 
indicate  relatively  weak  performance  by  MRand,  Stned,  and  Valstar .  The  first  plot  in 
Figure  4.3  shows  that  TMCBNxs  stochastically  dominated  hy  MRand,  Smed,  and  Valstar, 
while  the  second  shows  the  mixed  results  of  TMCBN  and  the  other  four  CEP  estimators 
under  consideration: 


Figure  4.3  Overall  CDF  Plots  Based  on  Our  Design  Point  Results 
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Figure  4.3  Overall  CDF  Plots  Based  on  Our  Design  Point  Results  (continued) 


As  previously  mentioned,  it  was  clear  that  virtually  all  design  points  with  a  bias  of 
4.0a  were  not  valid  for  MRand.  To  try  to  clarify  MRand 's  performance,  we  next 
constructed  a  CDF  plot  for  all  of  the  design  points  except  those  with  bias  4.0a; 


Figure  4.4  Overall  CDF  Plot  for  All  Design  Points  Except  Those  With  Bias  4.0a 


The  CDF  plot  in  Figure  4.4  still  shows  the  same  three  estimators  (including  MRand) 
stochastically  dominating  TMCBN.  The  results  of  this  plot  indicate  that  MRand 's 
relatively  weak  performance  is  not  entirely  driven  by  design  points  with  bias  greater  than 
4.0a. 


To  determine  which  factors  appeared  most  significant,  we  produced  these 
correlation  tables  for  our  design  point  results  using  Statistix  software  [1985]  : 


CORRELATIONS  (PEARSON) 

N  BIAS  P  SD_RATIO  THETA  SMED  ETHRIDGE  MRAND  VALSTAR  GRUBBS 

BIAS  -0.0000 

P  0.0000  -0.0400 


SD_RATIO 

0.0000  -0.0000 

0.0000 

THETA 

-0.0000  0.1027 

-0.2772 

-0.0000 

SMED 

-0.4425  -0.4521 

0.1465 

0.0219 

-0.0268 

ETHRIDGE 

-0.4794  -0.4584 

0.1705 

0.0088 

-0.0420 

0.8644 

MRAND 

-0.0856  0.1144 

0.0383 

0.0477 

0.0348 

-0.0188 

-0.0031 

VALSTAR 

-0.4525  -0.4152 

0.1707 

0.0893 

0.0063 

0.8571 

0.8667 

-0.0119 

GRUBBS 

-0.4862  -0.4120 

0.1496 

0.0763 

0.0017 

0.8694 

0.8963 

-0.0037 

0.9854 

RAYLEIGH  -0.4454  -0.4629 

0.1589 

0.0640 

-0.0173 

0.8957 

0.9340 

-0.0079 

0.9515  0.9645 

CASES  INCLUDED  1 100  MISSING  CASES  0 


CORRELATIONS  (PEARSON) 

N  BIAS  P  SD_RATIO  THETA  NUMERICAL 

BIAS  0.0176 

P  -0.0027  -0.0399 

SD_RATIO  0.0032  -0.0074  -0.0030 

THETA  0.0026  0.1028  -0.2809  0.0056 

NUMERICAL -0.3580  -0.2018  0.0826  0.1197  0.0214 

TMCBN  -0.3135  -0.1204  0.0643  0.1129  0.0315  0.9710 

CASES  INCLUDED  1088  MISSING  CASES  12 

Table  4.4  Statistix  Correlation  Tables  for  the  Design  Point  Results 


When  examining  these  results,  note  that  the  correlation  between  our  factors  is  not 
always  zero.  Because  of  the  symmetries  explained  in  Section  3.1  we  used  only  positive 
correlation  levels  along  the  X-axis  (where  0  =  0);  therefore  bias  and  correlation  appears 


correlated  to  0  in  the  upper  table.  In  the  lower  table,  the  twelve  design  points  that  would 
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not  converge  for  Numerical  and  TMCBN  were  not  considered  and  account  for  the 
nonzero  correlations  between  factors. 

The  correlation  between  either  sample  size  or  bias  and  any  of  the  eight  CEP 
estimators  tends  to  be  around  -0.4,  while  the  correlation  between  any  of  the  remaining 
factors  and  any  of  the  eight  CEP  estimators  falls  into  a  range  of  [-0.0485,  0.1707].  From 
these  results,  sample  size  and  bias  are  clearly  the  factors  most  strongly  influencing  the 
MSRE  scores.  This  judgment  corresponds  to  that  of  Tongue,  who  also  independently 
determined  sample  size  and  bias  to  be  the  most  crucial  factors  [Tongue  (1993),  page  5-7]. 

Having  resolved  sample  size  and  bias  to  be  the  most  significant  factors,  we  next 
analyzed  the  design  points  for  each  specific  sample  size  and  bias  factor  level.  Although 
each  case  was  analyzed,  we  group  cases  that  produced  identical  results  together  for 
presentation.  Based  on  their  weak  showing  in  the  overall  CDFs  and  MSRE  averages 
presented  earlier,  we  did  not  consider  MRawc/,  Smed,  or  Valstar  in  this  casewise  analysis. 


We  first  present  results  for  our  sample  size  cases,  beginning  with  error  statistics. 
We  received  essentially  identical  results  for  sample  sizes  fifteen,  nine,  and  six;  these  cases 
are  grouped  into  a  single  set  for  presentation: 


Sample  Size  =  15,  9,  or  6: 

Numerical 

TMCBN 

Average  MSRE 

0.0269 

0.024 

0.0269 

0.023 

0.0212 

Corresponding  MRE  Approximation 

0.1639 

0.1548 

Em 

0.1458 

AE  (True  CEP  was  100) 

±16.386 

±15.483 

±16.392 

±15.177 

±14.575 

Sample  Size  =  3: 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0787 

0.0844 

0.0756 

0.1017 

0.0983 

Corresponding  MRE  Approximation 

0.2805 

0.2905 

0.275 

0.3189 

0.3136 

AE  (True  CEP  was  100) 

±28.047 

±31.892 

±31.356 

Table  4.5  Statistics  for  Design  Point  Sample  Size  Cases 
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The  statistics  for  our  two  sample  size  cases  indicate  that  based  on  our  MSRE  results  the 
five  considered  CEP  estimators  produce  errors  of  approximately  14-17%  for  sample  sizes 
fifteen,  nine,  and  six;  this  rate  jumps  to  around  27-32%  at  sample  size  three. 

We  next  present  a  CDF  plot  for  each  of  these  sample  size  sets; 


Figure  4.5  CDF  Plots  Based  on  Our  Design  Point  Results  (n  =  3) 
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Figure  4.6  CDF  Plot  Based  on  Our  Design  Point  Results  (n  =  15,  9,  or  6) 


As  seen  in  the  CDF  plots  in  Figures  4.5  and  4.6,  TMCBN  appears  best  for  sample 
sizes  fifteen,  nine,  and  six,  while  Rayleigh  and  Ethridge  are  stochastically  dominated  at 
sample  size  three.  Bear  in  mind  when  observing  these  plots  that  a  stochastically  dominated 
distribution  indicates  good  performance,  since  high  MSRE  is  undesired. 

We  next  examine  the  design  point  results  classified  according  to  bias.  The  design 
points  with  bias  0,  0.5g,  or  l.Oo  yielded  similar  results  and  are  grouped  together  for 
presentation.  Like  the  sample  size  cases,  we  first  present  statistics  related  to  the  design 
points  classified  by  bias  followed  by  CDF  plots. 
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Bias  =  0, 0.5a,  1.0a 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0581 

0.0574 

0.0575 

0.0507 

Corresponding  MRE  Approximation 

0.241 

0.2396 

0.2251 

AE  (True  CEP  was  100) 

+24.095 

Bias  =  2.0a 

Ethridge 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0246 

0.0288 

0.0283 

Corresponding  MRE  Approximation 

0.1697 

0.1681 

AE  (True  CEP  was  100) 

±17.175 

±16.949 

±15.684 

±16.969 

±16.808 

Bias  =  4.0a 

Ethridge 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0094 

0.0086 

0.0096 

0.02 

0.0269 

Corresponding  MRE  Approximation 

0.0979 

0.1413 

0.1641 

AE  (True  CEP  was  100) 

Table  4.6  Statistics  for  Design  Point  Bias  Cases 


From  these  bias  statistics,  it  is  clear  that  the  margin  of  difference  in  the  expected 
error  is  relatively  small  for  design  points  where  bias  was  less  than  4.0a.  The  expected 
error  ranges  from  around  22-24%  for  bias  (0,  0.5a,  1.0a)  and  around  15-17%  for  design 
points  with  bias  =  2.0a.  The  MSRE  average  values  (and  thus  the  expected  error)  of  the 
two  numerical  estimators  Numerical  and  IMCBN,  however,  appear  substantially  higher  at 
bias  4.0a  than  the  other  three  considered  estimators. 

We  next  constructed  CDF  plots  for  each  of  these  three  bias  cases.  These  plots  are 
displayed  in  Figure  4.7: 
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Design  Points  With  Bias  =  2.0o; 


Design  Points  With  Bias  =  4.0a: 


Figure  4.7  CDF  Plots  Based  on  Design  Point  Bias  Cases 
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According  to  the  CDF  plots  shown  in  Figures  4.7  and  4.8,  TMCBN  appeared  to 
perform  best  for  bias  levels  of  1  .Og  and  under  while  Rayleigh  appeared  to  perform  best  for 
bias  2.0g.  At  bias  4.0a,  Ethridge,  Grubbs,  and  Rayleigh  appeared  to  outperform  the  two 


numerical  estimators. 

While  our  CDF  plots  based  on  sample  size  or  bias  cases  give  an  indication  of  the 
strengths  and  weaknesses  of  the  considered  CEP  estimators,  there  is  conflicting  "overlap" 
for  specific  cases.  For  example,  7MC5A  appears  best  for  sample  size  fifteen  and  Rayleigh 
appears  best  for  bias  2.0a  -  which  is  best  for  the  specific  case  of  sample  size  15  and  bias 
2.0a?  To  allow  a  comparison  of  the  estimators  for  specific  sample  size  and  bias 
combinations,  we  next  present  tables  that  display  for  each  estimator  the  MSRE  and  AE 


results  for  each  sample  size/bias  case; 


Table  4.7  Average  MSRE  Results  for  Design  Point  Sample  Size/Bias  Combinations 


Tin 

bias 

Smed 

mmm 

MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

mCBNf 

15 

0 

0.0444 

0.0259 

0.0145 

0.0177 

0.017 

0.0268 

0.0167 

0.0163 

15 

0.0382 

0.023 

0.0177 

0.0229 

0.0266 

0.0168 

15 

BEa 

0.0245 

0.0178 

0.0178 

0.0228 

0.0146 

0.0176 

0.0131 

la 

2.0a 

0.0112 

0.0115 

0.0101 

0.0123 

0.0087 

0.0079 

0.008 

0.008 

4.0a 

0.0035 

0.0033 

0.0834 

0.0027 

0.0023 

0.0023 

0 

0.0597 

0.0455 

0.0286 

0.0316 

0.0462 

0.0311 

0.0268 

9 

0.5a 

0.0629 

0.0379 

0.0354 

0.0346 

0.0429 

0.0334 

0.0301 

9 

1.0a 

0.0464 

0.0308 

0.0341 

0.042 

0.0277 

0.0252 

9 

2.0(7 

0.0272 

0.019 

0.0177 

0.0223 

0.0171 

0.015 

0.0166 

0.0163 

9 

0.0065 

0.0061 

0.0053 

0.0052 

0.0068 

0.005 

6 

HB 

0.1135 

0.087 

0.069 

0.0881 

0.068 

6 

0.5a 

0.0929 

0.0693 

0.0631 

0.0779 

0.0645 

0.0731 

0.0627 

0.0531 

6 

1.0a 

0.059 

0.046 

0.049 

0.0591 

0.0447 

0.0436 

0.0421 

0.0385 

6 

2.0a 

0.0385 

0.0366 

0.0291 

0.0256 

0.0288 

0.0281 

6 

4.0a 

0.0109 

0.0094 

0.0091 

0.01 

0.0091 

0.0105 

3 

0 

0.2186 

0.1458 

0.1841 

0.1584 

0.1408 

0.1564 

0.1334 

3 

0,5a 

0.1945 

■wwiyU 

0.1517 

0.1641 

0.1402 

0,128 

0.152 

0.1302 

3 

1.0a 

0.1326 

0.0988 

0.168 

0.1239 

0.0926 

0.1034 

0.0927 

3 

2.0o 

0.0816 

MtfAkTM 

0.06 

0.0499 

0.0623 

0.0612 

3 

4.0a 

0.0241 

0.017 

0.0712 

0.1008 
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Table  4.8  Approximate  Estimation  Error  (AE)  Based  on  Design  Point  Average  MSRE 
Results  for  Each  Sample  Size/Bias  Case  (Actual  CEP  was  100) 


n 

bias 

Smed 

I 

MRcind 

Vatstar 

Grubbs 

Raytei^h 

Numerical 

.mOBN:: 

15 

0 

±21.071 

±16.093 

±12.042 

±13.304 

±13.038 

±16.371 

±12.923 

±12.767 

15 

0.5a 

±19.545 

±15.166 

±13.304 

±15.133 

±13.229 

±16.31 

±12.961 

±12.689 

15 

1.0a 

±15.652 

±13.342 

±13.342 

±13.266 

±11.446 

±11.269 

15 

2.0a 

±10.583 

±10.724 

±10.05 

±11.091 

±9.3274 

±8.8882 

±8.9443 

±8.9443 

15 

4.0a 

±5.9161 

±5.7446 

±28.879 

±5.1962 

±5 

±4.7958 

±4.7958 

9 

0 

±24.434 

±21.331 

±16.912 

±18.762 

±17.776 

±21.494 

±17.635 

±16.371 

9 

0.5a 

±25.08 

±19.468 

±18.815 

±21.095 

±18.601 

±20.712 

±18.276 

±17.349 

9 

±17.55 

±18.466 

±20.494 

±17.176 

±17.378 

±16.643 

±15.875 

El 

±13.784 

±13.304 

±14.933 

±13.077 

±12.247 

±12.884 

±12.767 

wm 

4.0a 

±8.0623 

±7.8102 

±39.446 

±7.2801 

±7.2111 

±8.2462 

±7.0711 

±7.4833 

6 

0 

±33.69 

±29.496 

±25.768 

±28.965 

±26.268 

±29.682 

±26.077 

±23.937 

6 

0.5a 

±30.48 

£26.325 

±25.12 

±27.037 

±25.04 

±23.043 

l.Oa 

±24.29 

±21.448 

±22.136 

±24.31 

±21.142 

±20.881 

±20.518 

±19.621 

±17.292 

±17.72 

±19.131 

±17.059 

±16 

±16.971 

±16.763 

6 

±9.798 

±86.539 

±9.6954 

±9.5394 

±10 

±9.5394 

±10.247 

3 

0 

±46.755 

±37.523 

±38.184 

±42.907 

±39.799 

±37.523 

±39.547 

±36.524 

3 

0.5a 

±44.102 

±35.426 

±38.949 

±40.509 

±37.443 

±35.777 

Hb38.987 

±36.083 

3 

l.Oa 

±36.414 

±31.432 

±40.988 

±35.199 

±32.078 

±30.43 

±32.156 

±30.447 

3 

2.0a 

±28.566 

±24 

±97.811 

±22.338 

±24.96 

±24.739 

3 

±13.601 

±503.65 

±13.491 

±13.229 

±13.038 

±26.683 

±31.749 

Thus  far  in  our  design  point  presentation  we  have  only  presented  MSRE  results.  In 
Figures  4.8  through  4.11,  we  display  the  MRE  and  VRE  results  which  correspond  to  the 
MSRE  values  presented  in  Table  4.6.  For  each  sample  size  and  bias  case,  these  plots 
show  the  MRE  (labeled)  along  with  a  range  of  values  plus  or  minus  two  standard 
deviations  from  the  MRE.  This  range  indicates  the  VRE  scores  for  each  case,  since  one 
standard  deviation  from  the  MRE  is  equal  to  the  square  root  of  the  VRE. 

In  these  plots,  note  that  the  range  of  values  two  standard  deviations  below  the 
MRE  extends  past  zero  on  the  low  side.  These  negative  values  are  due  to  our  assumption 
that  the  MRE  scores  for  each  estimator  follow  a  normal  distribution.  Obviously  we  had 
no  negative  RE  scores  and  these  negative  portions  of  the  ranges  can  be  ignored. 
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Sample  Size  =15,  Bias  =  0.5a 


0.8  -- 

0.6  - 

0.4  -* 

♦  MRE-(-2SD 

0.2  -- 

■  MRE 

0  -■ 

AMRE-2SD 

-0.2  -■ 

Ethridge 


Sample  Size  =15,  Bias  =  2.0a 


Figure  4.8  Design  Point  RE  Plots  for  Each  Bias  Case  for  Sample  Size  Fifteen 
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Figure  4.9  Design  Point  RE  Plots  for  Each  Bias  Case  for  Sample  Size  Nine 
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Sample  Size  =  6,  Bias  =  0 


Sample  Size  =  6,  Bias  =  2.0a 
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Figure  4.10  Design  Point  RE  Plots  for  Each  Bias  Case  for  Sample  Size  Six 
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Figure  4.11  Design  Point  RE  Plots  for  Each  Bias  Case  for  Sample  Size  Three 
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These  plots  indicate  the  following  about  our  design  point  MRE  and  VRE  results: 

1.  The  difference  in  MRE  and  VRE  was  marginal  for  most  cases.  Only  MRand's  poor 
scores  at  high  bias  cases  clearly  stood  out,  and  this  was  primarily  due  to  using  the 
estimator  for  samples  for  which  it  was  not  valid. 

2.  Both  MRE  and  VRE  tend  to  increase  at  smaller  sample  sizes  and  smaller  bias  levels; 
conversely,  these  MOEs  tend  to  decrease  for  larger  sample  sizes  and  larger  bias  levels. 

4.3  Comparison  With  Previous  Studies 

We  next  compare  our  design  point  results  with  those  from  the  previous  CEP 
comparison  studies  ofElder  (1986),  Puhek  (1992),  and  Tongue  (1993). 

1.  Elder:  Elder  compared  MRand,  Gruhbs,  and  the  Grubbs-Patniak/  Wilson-Hilferty 
estimator  in  his  simulation  experiment.  Based  on  the  number  of  design  points  for  each 
estimator  that  yielded  the  minimum  RE,  Elder  concluded  that  Grubbs  had  equal  or  better 
results  Xh'axi  MRand  for  most  bias  and  ellipticity  cases  [Elder  (1986),  4-2  through  4-4]. 
Our  design  point  results  indicate  a  similar  conclusion: 


Figure  4.12  Our  Overall  Design  Point  Results  for  Grubbs  and  MRand 
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2.  Puhek:  Puhek  (1992)  considered  Grubbs,  MRand,  Ethridge,  and  Rayleigh  in  his 
comparison  study,  concluding  that  Rayleigh  generally  dominated  the  comparison.  He 
rated  Rayleigh  superior  for  all  design  points  except  those  where  bias  equaled  1.5a,  where 
Grubbs  and  Rayleigh  were  judged  to  have  equal  performance  [Puhek(1992),  page  4-8]. 

Our  results  did  not  show  such  dominance  by  Rayleigh.  According  to  our  results 
for  sample  sizes  fifteen,  nine  and  six,  Grubbs  appears  best  for  these  four  estimators,  as 
indicated  in  the  CDF  plot  below: 


Figure  4.13  Our  Design  Point  Results  for  the  Estimators  Considered  by  Puhek  at  Sample 

Sizes  Fifteen,  Nine,  and  Six 


We  can  only  speculate  why  our  results  were  much  different  from  Puhek's.  One 
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possible  explanation  is  the  fact  that  our  experimental  designs  were  different.  Puhek  used 
only  positive  correlation  levels  and  the  largest  bias  he  considered  was  2a.  In  addition, 
Puhek's  conclusions  were  based  strictly  on  MRE.  Even  considering  only  the  equivalent 
portion  of  our  results  leads  to  different  conclusions,  however. 

Another  possible  reason  for  this  disparity  is  the  fact  that  at  each  design  point 
Puhek  only  had  one  relative  error  estimate  for  each  of  the  four  CEP  estimators  under 
consideration.  With  the  replications  used  in  our  experiment,  each  design  point  has  an 
average  relative  error  estimate.  The  potential  for  variance  was  thus  much  more  prevalent 
in  Puhek's  study  and  may  have  led  to  misleading  results. 

3.  Tongue:  Tongue's  study  consisted  of  two  comparison  experiments.  In  the  first,  he 
compared  the  four  estimators  fi’om  Puhek's  study  along  with  CBN,  a  numerical  integration 
CEP  estimator  discussed  in  Section  2.3.  He  then  repeated  the  experiment,  replacing  CBN 
with  his  empirically  derived  TMCBN  estimator.  It  is  this  second  experiment  that  we 
compare  our  overall  design  point  results  to. 

Tongue's  results  are  quite  similar  to  ours.  As  previously  mentioned,  he  determined 
sample  size  and  bias  to  be  the  two  most  significant  factors  in  his  simulation  experiment,  in 
agreement  with  our  results.  For  most  sample  size  and  bias  combinations.  Tongue 
concluded  that  his  TMCSA" estimator  performed  best  [Tongue  (1993),  page  6-13]. 

Tongue  constructed  a  sample  size/bias  decision  grid  based  on  MRE 
[Tongue(1993),  page  6-13].  Comparing  Tongue's  grid  to  our  grid  which  displays  the 
estimator  -with  the  minimum  average  MSRE  for  each  sample  size/bias  case  demonstrates 
the  similarity  in  results: 
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Our  Grid  Showing  the  CEP  Estimator  With  the  Minimum  Average  MSRE, 
Based  on  Our  Design  Point  Results 
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Tongue's  Overall  Decision  Grid  Based  on  his  MRE  Results 


Figure  4.14  Our  Design  Point  Grid  Contrasted  Against  Tongue's  Decision  Grid 


4.4  The  Sample  Analysis  Set  Results 

In  this  section,  we  repeat  the  presentation  sequence  used  earlier  in  Section  4.2  for 
the  sample  analysis  set  results.  Before  we  proceed,  we  first  remind  the  reader  how  the 
sample  analysis  sets  were  constructed.  When  we  use  the  word  "sample"  in  this  thesis,  we 
refer  to  a  set  of  either  three,  six,  nine,  or  fifteen  (x,y)  coordinates,  depending  on  the 
sample  size.  At  each  of  our  1,100  design  points,  we  created  ten  simulated  samples.  The 
union  of  these  11,000  samples  is  identical  to  the  union  of  the  11,000  samples  that  are 
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elements  of  some  sample  analysis  set.  The  samples  are  just  classified  differently  in  each 
case.  The  sample  elements  for  each  design  point  were  classified  according  to  the 
population  parameters  used  to  generate  the  samples,  while  the  sample  analysis  sets  are 
categorized  according  to  the  sample  statistics  of  each  sample. 

Just  as  we  did  for  the  design  point  results,  we  begin  our  presentation  of  the  sample 
analysis  set  results  by  displaying  the  overall  histograms  for  each  CEP  estimator 


considered: 


Figure  4.15  Overall  Histograms  Based  on  the  Sample  Analysis  Set  Results 
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Figure  4.15  Overall  Histograms  Based  on  the  Sample  Analysis  Set  Results  (continued) 
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Figure  4.15  Overall  Histograms  Based  on  the  Sample  Analysis  Set  Results  (continued) 
These  histograms  basically  mirrored  the  design  point  results.  For  Numerical  and 
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TMCBN,  however,  the  number  of  sample  analysis  sets  with  relatively  high  MSRE  is 
proportionally  larger  than  the  number  of  design  points  with  relatively  high  MSRE. 

We  next  present  statistics  and  CDF  plots  for  our  overall  sample  analysis  set  results: 


Statistics 
Average  MSRE 
Corresponding  MRE 
Approximation 


AE  (True  CEP  was  100)  ±19.81  ±16.86 


MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

0.6646 

0.0325 

0.0267 

0.0273 

0.9827 

1.2813 

0.8152 

0.1804 

0.1633 

0.1651 

0.9913 

1.132 

±81.524 

±18.04 

±16.33 

±16.509 

±99.133 

Bllclll 

The  upper  graph  in  Figure  4.16  shows  that  Grubbs  is  stochastically  dominated  by 
MRand,  Smed,  and  Valstar,  while  the  lower  graph  displays  the  mixed  results  of  Grubbs 
and  the  other  four  CEP  estimators  under  consideration.  Note  in  this  lower  graph  that  the 
two  numerical  estimators  "separate"  from  the  others  around  probability  0.9.  Based  on  this 
CDF  plot,  the  MSRE  statistics,  and  the  histograms  presented,  clearly  there  are  some 
sample  analysis  set  cases  where  Numerical  and  TMCBN  performed  more  poorly  than  they 
had  for  the  corresponding  design  point  cases. 

As  we  had  done  earlier  for  the  design  point  results,  we  produced  correlation  tables 
results  to  determine  which  factors  appeared  most  significant  for  our  sample  analysis  sets: 


CORRELATIONS  (PEARSON) 

N  BIAS  P  SD_RATIO  SMED  ETHRIDGE  MRAND  VALSTAR  GRUBBS 

BIAS  0.0000 

P  0.0000  0.0000 


SD  RATIO 

0.0000  -0.0000 

-0.0000 

SMED 

-0.3864  -0.2744 

0.2328 

0.0700 

ETHRIDGE 

-0.4596  -0.3260 

0.2457 

0.0694 

0.9062 

MRAND 

-0.1523  0.2089 

0.0623 

-0.0534 

0.0210 

0.0139 

VALSTAR 

-0.4280  -0.1848 

0.2799 

0.1491 

0.8996 

0.8798 

0.0322 

GRUBBS 

-0.4639  -0.1929 

0.2254 

0.1354 

0.9092 

0.9082 

0.0389 

0.9828 

RAYLEIGH 

-0.4173  -0.3094 

0.2224 

0.1238 

0.9210 

0.9419 

0.0069 

0.9292 

CASES  INCLUDED  400  MISSING  CASES  0 


CORRELATIONS  (PEARSON) 


N 

BIAS 

P 

SD_RATIO 

NUMERICAL 

BIAS 

0.0047 

P 

-0.0187 

-0.0072 

SD  RATIO 

-0.0090 

-0.0210 

-0.0105 

NUMERICAL 

-0.0838 

0.1045 

0.0264 

-0.0571 

TMCBN 

-0.0794 

0.0995 

0.0191 

-0.0545 

0.9947 

CASES  INCLUDED  384  MISSING  CASES  16 


Table  4.10  Statistix  Correlation  Tables  for  the  Sample  Analysis  Set  Results 
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Unlike  the  1,100  design  points,  the  400  sample  analysis  sets  are  balanced  Avith 
regard  to  the  sample  statistic  ranges  which  define  the  sets.  Therefore,  in  the  upper  table 
all  correlation  values  are  zero  between  these  sample  statistic  range  "factors."  The  16 
nonconvergent  cases  for  Numericcil  and  TMCBN  account  for  the  nonzero  correlation 
values  between  the  sample  statistic  ranges  in  the  lower  table. 

Considering  the  results  from  these  tables,  sample  size  and  sample  bias  appeared  to 
be  the  most  significant  factors,  just  as  sample  size  and  bias  were  most  significant  for  our 
design  point  results.  There  were  differences,  however.  Sample  correlation  appeared  more 
significant  for  our  sample  analysis  set  results  than  correlation  had  for  our  design  point 
results,  ranging  from  Q.0\9\(TMCBN)  up  to  0.2799  (Valstar).  We  also  noted  that  for  our 
sample  analysis  set  results  the  two  numerical  estimators  appeared  relatively  uncorrelated 
to  any  factor. 

Based  on  these  correlation  table  results,  we  decided  to  examine  the  sample  analysis 
sets  according  to  specific  sample  size  and  sample  bias  ranges  in  a  manner  similar  to  our 
analysis  of  the  design  point  results.  Due  to  their  relatively  poor  performance  in  the  overall 
CDF  plots  presented  in  Figure  4.16  and  the  statistics  presented  in  Table  4.8,  MRand, 
Smed,  and  Valstar  were  not  considered  in  our  casewise  analysis  of  the  sample  analysis 
sets. 

We  begin  our  examination  of  the  sample  size  case  results  by  presenting  for  each 
estimator  the  average  MSRE  and  the  MRE  and  AE  approximations  that  correspond  to 
these  MSRE  averages.  Because  of  their  similar  results,  we  present  the  sample  size  cases 
of  fifteen,  nine,  and  six  as  one  group; 
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Sample  Size  =  15, 9,  or  6 

Ethridge 

Grubbs 

Average  MSRE 

0.0213 

0.0184 

0.0208 

0.0184 

0.0172 

Corresponding  MRE 
Approximation 

0.1459 

0.1356 

0.1444 

0.1356 

0.1311 

AE  (True  CEP  was  100) 

±14.589 

±13.558 

±14.435 

±13.564 

±13.109 

Sample  Size  =  3 

Grubbs 

Numerical 

TMCBN 

Average  MSRE 

0.0498 

0.0515 

0.0465 

4.4268 

5.7961 

Corresponding  MRE 
Approximation 

0.2232 

0.227 

0.2156 

2.104 

2.4075 

AE  (True  CEP  was  100) 


±22.315 


±22.699 


±21.565 


±210.4 


±240.75 


Our  CDF  plots  correspond  to  the  statistics  presented  earlier  for  these  two  sample 
size  sets.  TMCBN  performed  marginally  better  at  sample  sizes  fifteen,  nine  and  six,  while 
at  sample  size  three  TMCBN,  Rayleigh  and  Ethridge  performed  best. 

We  next  present  statistics  and  CDF  plots  relating  to  sample  bias  range  cases.  Our 
CDF  plots  for  the  sample  bias  range  cases  under  2.75  c  provided  no  insights;  for  this 
reason  we  present  these  cases  in  a  single  plot. 


Sample  Bias 
=  [0,  0.75cl 

Ethridge 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0383 

0.0321 

0.0384 

0.0307 

0.0269 

Corresponding  MRE 
Approximation 

0.1958 

0.1793 

0.196 

0.1753 

0.1639 

AE  (True  CEP  was  100) 

±19.577 

±17.926 

±19.598 

Sample  Bias 
=  (0.755, 1.25  o] 

Ethridge 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.035 

0.0289 

0.0309 

0.0289 

0.0268 

Corresponding  MRE 
Approximation 

0.1872 

0.1699 

0.1758 

0.1701 

0.1637 

AE  (True  CEP  was  100) 

±18.718 

±16.994 

±17.576 

±17.014 

±16.366 

Sample  Bias 
=  (1.255,2.755] 

Ethridge 

Grubbs 

Rayleigh 

Numerical 

TMCBN 

Average  MSRE 

0.0261 

0.0298 

0.0257 

0.034 

0.0317 

Corresponding  MRE 
Approximation 

0.1615 

0.1727 

0.1602 

0.1843 

0.1781 

AE  (True  CEP  was  100) 

±16.148 

±17.271 

±16.019 

Sample  Bias 
>  2.755 

Ethridge 

Grubbs 

Rayleigh 

Average  MSRE 

0.0142 

0.0158 

0.0141 

5,0926 

Corresponding  MRE 
Approximation 

0.1192 

0.1258 

0.1185 

12S61 

AE  (True  CEP  was  100) 

±11.92 

±12.582 

±11.855 

±196.91 

±225.67 

Table  4.12  Statistics  for  Sample  Analysis  Set  Sample  Bias  Cases 
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Figure  4.19  Sample  Analysis  Set  CDF  Plots  Based  on  Sample  Bias  Range 


There  appears  little  disparity  between  the  statistics  and  CDF  plots  of  the 
competing  estimators  except  for  the  case  of  sample  bias  greater  than  2.75 a,  where 
Rayleigh  appears  best  and  Numerical  and  ZMCBJV  perform  notably  poorly 

We  next  present  the  average  MSRE  and  the  AE  results  for  each  CEP  estimator 
which  correspond  to  each  sample  size/sample  bias  range  combination.  The  corresponding 
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design  point  tables  are  also  displayed  again  to  allow  comparison  between  the  two  results: 


Average  MSRE  Results  for  Sample  Analysis  Set  Samp! 


■Il'v-: 

bias 

Smed 

W!mm 

MRdnd 

Walstar 

Grubbs 

Numerical 

!mmsm 

15 

[0,0.75  E] 

0.0324 

0.0193 

0.0129 

0.0161 

0.0139 

0.0211 

15 

(0.75  CT,  1.25  a] 

0.0224 

0.0157 

0.0157 

0.0208 

0.0131 

0.0171 

0.0119 

0.0114 

15 

(1.25  a, 2.75  c] 

|j|jyy 

0.0114 

0.0112 

0.0139 

0.009 

0.0087 

mitmAm 

0.0078 

15 

(>2.75  a) 

0.0028 

0.0025 

0.004 

WiKMM 

0.0024 

9 

[0,0.75  c] 

0.0255 

0.0233 

0.032 

0.0226 

0.0207 

(0.75  c, 1.25  c] 

DKIX^ 

0.0258 

0.0226 

0.0287 

0.0209 

0.0232 

0.0196 

0.0188 

(1.25  c, 2.75  c] 

0.0283 

0.0172 

0.0248 

0.0202 

0.0171 

9 

(>2.75  c) 

0.0089 

0.0066 

0.1195 

0.0069 

0.0072 

0.0069 

0.0069 

6 

[0,0.75  E] 

0.0566 

0.0394 

0.0365 

0.046 

0.0356 

0.0318 

6 

(0.75  c,  1.25  c] 

0.0388 

0.029 

0.0355 

0.0279 

0.0322 

0.0264 

0.0244 

6 

(1.25c,2.75c] 

liltltfcfJ 

0.0426 

0.0305 

0.0266 

0.0293 

0.0257 

6 

0.0136 

0.516 

0.016 

0.0138 

0.0254 

0.0261 

3 

0.0554 

0.0442 

nmm 

0.0548 

0.0545 

0.0552 

0.0448 

3 

(0.75  c,  1.25  c] 

0.0767 

0.0598 

0.0534 

0.0637 

0.0536 

0.0511 

0.0633 

0.0574 

3 

(1.25  c, 2.75  c] 

0.0793 

0.0725 

0.0597 

0.0492 

0.0854 

0.0824 

3 

(>2.75  c) 

0.0453 

0.0338 

9.5637 

0.0428 

0.038 

0.0312 

18.374 

24.146 

e  Size/Sample  Bias  Combinations: 


Its  for  Design  Point 


Sample  Size/Bias  Combinations 


ri 

bias 

1 

. — 

StMd 

1 

MRand 

Kalstar 

Grubbs 

Rayleigh 

Numerical 

TMCBNm 

15 

0 

0.0444 

0.0259 

0.0145 

0.0177 

0.017 

0.0268 

0.0167 

0.0163 

15 

0.5a 

0.0382 

0.023 

0.0177 

0.0229 

0.0175 

0.0266 

0.0168 

0.0161 

15 

1.0a 

0.0245 

0.0178 

0.0178 

0.0228 

0.0146 

0.0176 

0.0131 

0.0127 

15 

0.0112 

0.0115 

0.0101 

0.0123 

0.008 

0.008 

15 

EE9 

0.0035 

0.0033 

0.0834 

0.0027 

0.0025 

0.0045 

0.0023 

0.0023 

9 

0 

0.0597 

0.0455 

0.0286 

0.0352 

0.0462 

0.0311 

0.0268 

9 

0.5a 

0.0629 

0.0379 

0.0354 

0.0445 

0.0429 

0.0334 

0.0301 

9 
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0.0464 

0.0308 

■atigw 

0.042 
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0.0302 

0.0252 

9 

rag 

0.019 

0.0223 

0.0163 

9 

4.0a 

0.0065 

0.0061 

0.1556 

0.0053 

0.0052 

0.0068 

0.005 

0.0056 

0 

0.1135 

0.087 

0.0664 

0.0839 

0.069 

0.0881 

0.068 

0.0573 

0.5c 

0.0929 

0.0693 

0.0631 

0.0779 

0.0645 

0.0731 

0.0627 

0.0531 

6 

LOa 

0.059 

0.046 

0.049 

0.0591 

0,0447 

0.0436 

0.0421 

0.0385 

6 

2.0c 

0.0385 

0.0299 

0.0366 

0.0288 

0.0281 

6 

4.0a 

0.0109 

0.0096 

0.0094 

0.0091 

0.0105 

3 

0 

0.2186 

0.1408 

0.1458 

0.1408 

0.1564 

0.1334 

B 

0.5a 

0.1945 

0.1255 

0.1517 

0.1641 

0.1402 

0.128 

0.152 

0.1302 

B 

0.0988 

0.168 

0.1239 

0.1029 

0.0926 

0.1034 

0.0927 

2.0a 

0.0816 

0.0576 

0.9567 

0.0716 

0.06 

0.0623 

0.0612 

3 

0.0185 

25.366 

0.0182 

0.0175 

0.017 

0.0712 

0.1008 

Table  4.13  Comparison  of  Design  Point  and  Sample  Analysis  Set  Average  MSRE  Results 
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Approximate  Estimation  Error  (AE)  Based 
Results  for  Each  Sample  Size/Sample 


bias 


[0,0.75  a] 


MRand 


+13.892  I  ±11.358 


on  Sample  Analysis  Set  Average  MSRE 
Bias  Case  (Actual  CEP  was  100): _ 


Valstar 


15  I  (0.75  a,  1.25  all  ±14.967  |  ±12.53 


15  I  (1.25 a, 2.75 a] 


±10.909 


(>2.75 a)  I  ±5.7446  ±5.3852  ±25.942 


[0,0.75  a]  ±21.656  ±17.861  ±14.457 


(0.75 a, 1.25 a]  ±19.157  |  ±16.062  |  ±15.033 


(1.25a,2.755]  I  ±16.823 


±9.434 


±23.791  ±21.587  |  ±17.916 


±15.969 


Grubbs; 

Rayleigh 

Numerical 

TMCBN 

±11.792 

±14.537 

±11.593 

±11.428 

±11.458 

±10.679 

±9.4636 

±8.8544 

±4.9518 

±6.2992 

±4.7666 

±4.9193 

±15.26 

±17.891 

±15.049 

±14.376 

±14.446 

±15.217 

±13.724 

(>2.75  a) 


[0,0.75  a] 


(0.75  a,  1.25  a] 


(1.25 a, 2.75 a]  |  ±18.868 


(>2.75  a) 


[0,0.75  a] 


(0.755,1.255]  ±27.695  ±24.454  ±23.108 


(1.255,2.75  a]  ±28.16  ±22.383  ±25.2 


(>2.755)  ±21.284  ±18.385  ±309.25 


±13.711 


±24.352  ±23.409  |  ±23.335 


±25.239  ±23.152 


±26.926  ±24.434  ±22.184 


±20.688  ±19.494  ±17.669 


±23.491 


±29.225 


±428.65 


±8.309 


±17.819 


±15.632 


±16.019 


±16.168 


±21.167 


±23.949 


±28.697 


±491.38 


Approximate  Estimation  Error  (AE)  Based  on 

_ Each  Sample  Size/Bias  Case 

n  I  bias  I  Smed  Ethridge  \  MRand  \  Valstar 


±21.071 


1.0a  ±15.652 


2.0a  ±10.583 


±5.9 


±24.434 


±25.08 


Design  Point  Average  MSRE  Results  for 
(Actual  CEP  was  100): _ _ 


4.0a  ±8.0623 


0  ±33.69 


0.5a  ±30.48 


0.5a  ±44.102 


1.0a  ±36.414 


Ethridge 

MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

TMCBfm 

±16.093 

±12.042 

±13.304 

±13.038 

±16.371 

±12.923 

±12.767 

±15.166 

±13.304 

±16.31 

±12.961 

±12.689 

±13.342 

±13.342 

±15.1 

±12.083 

±13.266 

±11.446 

±11.269 

±10.724 

±10.05 

±11.091 

±9.3274 

±8.8882 

±8.9443 

±8.9443 

±5.7446 

±28.879 

±5 

±6.7082 

±4.7958 

±4.7958 

±21.331 

±16.912 

±18.762 

±17.776 

±21.494 

±17.635 

±16.371 

±19.468 

±18.815 

±21.095 

±18.601 

±20.712 

±18.276 

±17.349 

±17.55 

±18.466 

±16.643 

±15.875 

±13.784 

±13.304 

±14.933 

±12.767 

±7.8102 

±39.446 

±7.2801 

±7.2111 

±8.2462 

±7.0711 

±7.4833 

±29.496 

±25.768 

±28.965 

±26.268 

±29.682 

±26.071 

±23.937 

±26.325 

±25.12 

±27.911 

±25.397 

±27.037 

±25.04 

±23.043 

±21.448 

±22.136 

±24.31 

±21.142 

±20.881 

±20.518 

±19.621 

±17.292 

±17.72 

±19.131 

mem 

±16.971 

±16.763 

±9.798 

±86.539 

±9.6954 

±9.5394 

±10 

±9.5394 

±10.247 

±37.523 

±38.184 

±37.523 

±39.547 

±36.524 

±35.426 

±38.949 

±40.509 

±37.443 

±35.777 

±38.987 

±36.083 

±31.432 

±40.988 

±35.199 

±32.078 

±30.43 

±32.156 

±30.447 

±24 

±97.811 

msmiEsm 

±22.338 

±24.96 

±24.739 

±13.601 

±503.65 

±13.491 

±13.229 

±13.038 

±26.683 

±31.749 

Table  4.14  Comparison  of  Design  Point  and  Sample  Analysis  Set  AE  Results 
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We  conclude  our  presentation  of  our  sample  analysis  set  results  by  displaying  RE 
plots  which  correspond  to  those  presented  for  the  design  point  results  in  Figures  4.8 


through  4.11; 


Figure  4.20  Sample  Analysis  Set  RE  Plots  for  Each  bias  Case  for  Sample  Size  Fifteen 
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Sample  Size  =  9,  Sample  Bias  =  [0,  0.75  a  ] 
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Figure  4.21  Sample  Analysis  Set  RE  Plots  for  Each  bias  Case  for  Sample  Size  Nine 


Sample  Size  =  6,  Sample  Bias  =  [0,  0.75  a] 
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Figure  4.22  Sample  Analysis  Set  RE  Plots  for  Each  bias  Case  for  Sample  Size  Six 
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Sample  Size  =  3,  Sample  Bias  =  [0,  0.75  a  ] 
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Figure  4.23  Sample  Analysis  Set  RE  Plots  for  Each  bias  Case  for  Sample  Size  Three 


These  plots  show  that  our  sample  analysis  set  results  closely  resembled  those  of 
our  design  points.  Again,  the  difference  in  MRE  and  VRE  scores  for  the  eight  CEP 
estimators  considered  was  marginal  for  most  cases.  The  one  noticeable  difference  in  our 
sample  analysis  set  results,  however,  is  the  increased  MRE  and  VRE  of  the  two  numerical 
estimators  for  sample  size  three  and  sample  bias  greater  than  2.75  a . 

This  concludes  our  presentation  of  the  results  of  our  simulation  experiment.  The 
overall  and  casewise  statistics,  CDF  plots,  and  MRE  plots  shown  in  this  chapter  provide 
the  basis  for  our  recommendations  presented  in  Chapter  5. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 

In  this  final  chapter,  we  summarize  our  findings  and  make  recommendations  based 
on  the  results  of  the  simulation  experiment.  We  first  highlight  the  more  notable 
observations  made  during  the  experiment.  We  then  address  how  our  results  relate  to  the 
objectives  listed  in  Section  1.4  and  finally  conclude  by  presenting  suggestions  for  further 
study. 

1.  Nonparametric  CEP  Estimators:  Smith  (1982)  claimed  that  nonparametric  CEP 
estimators  were  not  suitable  for  small  samples.  We  tested  this  claim  by  considering  the 
nonparametric  sample  median  estimator.  Based  on  the  weak  performance  of  Smed,  we 
concur  with  Smith. 

2.  Numerical  Integration  Methods:  Elder  [1986,  page  5-4]  and  Puhek  [1992,  page  2-7] 
assumed  (but  did  not  test)  that  a  numerical  integration  CEP  estimator,  while  requiring 
excessive  computation  time,  was  generally  always  superior  to  a  non-numerical  CEP 
estimator.  While  Numerical  and  TMCBN  do  require  more  computation  time  than  CEP 
estimators  fi-om  other  categories,  it  is  hardly  prohibitive  on  today's  Pentium  equipped 
computers  (a  sample  of  15  coordinates  is  usually  computed  in  under  a  minute).  Our 
results  and  the  earlier  results  of  Tongue  (1993)  show,  however,  that  there  are  numerous 
instances  where  the  non-numerical  estimators  outperformed  Numerical  or  TMCBN. 

Tongue  claimed  that  CBN  (Numerical)  and  TMCBN  dominated  for  sample  sizes 
greater  than  ten.  Our  sample  analysis  set  MSRE  results  for  sample  size  fifteen  supports 
this  assertion. 

3.  MOEs:  With  regard  to  the  MOEs  used  in  our  experiment,  we  noticed  in  our  results 
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that  the  CEP  estimator  which  was  best  for  MSRE  was  also  usually  best  for  MRE  and/or 
VRE.  Since  MSRE  combines  aspects  of  both  MRE  and  VRE,  we  recommend  MSRE  if 
only  one  MOE  is  used  in  a  comparison  of  CEP  estimators. 

4.  Estimators  Not  Considered  in  Previous  Studies:  Two  of  the  CEP  estimators  compared 
in  this  study,  Smed  and  Valstar,  were  not  considered  in  the  previous  works  of  Elder 
(1986),  Puhek  (1992),  or  Tongue  (1993).  Based  on  both  our  design  point  and  sample 
analysis  set  results,  neither  of  these  estimators  was  highly  competitive. 

5.1  Results  Based  on  the  Thesis  Objectives 

In  this  section,  we  revisit  the  three  thesis  objectives  first  presented  in  Section  1.4. 
Objective  1:  The  first  of  our  objectives  was  to  develop  a  tool  which  CEP  analysts  could 
use  to  determine  which  CEP  estimator  to  use  for  small  samples  (15  or  less)  for  a  given  set 
of  conditions.  We  base  our  recommendations  on  the  precision  that  required  for  the  CEP 
estimate. 

If  maximum  precision  is  required,  we  recommend  the  follovwng  decision  grid  based 
on  the  best  average  MSRE  value  for  each  sample  analysis  set  sample  size/sample  bias 
combination; 


15 

9  6 

3 

[0,  0.755] 

(0.755,1.255] 

(1.255,2.755] 

(>2.755  ) 

TMCBN 

Ethric^e 

Figure  5.1  Recommended  Decision  Grid  for  Maximum  Precision 
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We  base  our  recommendations  on  our  sample  analysis  set  results  rather  than  the 
design  point  results  because  the  sample  analysis  sets  model  the  real  life  scenario  of 
estimating  CEP  based  on  only  sample  statistics. 

If  the  decision  maker  is  willing  to  tolerate  relatively  small  increase  in  the  amount  of 
anticipated  error,  we  can  recommend  more  flexible  choices.  Recall  that  for  a  given  case 
the  AE  for  a  given  estimator  refers  to  the  approximate  error  one  can  expect  from  that 
estimator  based  on  it's  MSRE.  Let  MAE  represent  the  minimum  AE  based  on  the  sample 
analysis  set  values  from  Table  4.8  for  each  sample  size/sample  bias  case.  In  the  next  three 
grids,  we  present  CEP  estimators  which  have  AE  values  within  0.01  *CEP,  0.02*CEP,  and 
0.05*CEP  of  the  MAE  respectively; 
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Figure  5.2  CEP  Estimators  Which  Have  AE  Values  Within  0.01*CEP  of  the  MAE. 
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Figure  5.3  CEP  Estimators  Which  Have  AE  Values  Within  0.02*CEP  of  the  MAE, 
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We  next  recommend  decision  grids  for  the  event  in  which  the  CEP  analyst  does 
not  have  access  to  a  personal  computer.  The  Numerical  and  TMCBN  estimators  both 
require  a  either  a  computer  program  manually  written  in  a  language  such  as  FORTRAN  or 
special  software  such  as  MathCAD  to  perform  numerical  approximation.  In  a  situation 
where  the  CEP  analyst  does  not  have  access  to  a  personal  computer  and  must  estimate  the 
CEP  "by  hand",  we  recommend  the  following  decision  grids  based  on  our  sample  analysis 
set  MSRE  and  AE  results: 
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Figure  5.5  Recommended  Decision  Grid  for  Maximum  Precision, 
Numerical  Methods  Excluded 
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Figure  5.6  CEP  Estimators  Which  Have  AE  Values  Within  0.01*CEP  of  the  MAE, 

Numerical  Methods  Excluded 
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Figure  5.7  CEP  Estimators  Which  Have  AE  Values  Within  0.02*CEP  of  the  MAE, 

Numerical  Estimators  Excluded 
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Figure  5.8  CEP  Estimators  Which  Do  Not  Have  AE  Values  Within  0.05*CEP  of  the 

MAE,  Numerical  Estimators  Excluded 


Finally,  in  addition  to  not  having  access  to  a  personal  computer,  an  analyst  could 
face  a  situation  where  they  not  have  access  to  the  chi-square  distribution  tables  required 
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for  the  Grubbs  computation.  In  this  case,  the  analyst  only  has  the  capability  to  perform 
the  necessary  calculations  for  Smed,  Ethridge,  MRand,  Valstar,  and  Rayleigh.  We 
recommend  the  following  decision  grids  based  on  our  sample  analysis  set  MSRE  and  AE 
results  for  this  instance: 
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(0.75  a,  1.25  5] 
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Ethridge 

Figure  5.9  Recommended  Decision  Grid  for  Maximum  Precision,  Grubbs,  Numerical, 

and  7MC5iV  Excluded 
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Figure  5.10  CEP  Estimators  Which  Have  AE  Values  Within  0.01*CEP  of  the  MAE, 
Grubbs,  Numerical,  and  7MCBV  Excluded 
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Figure  5.11  CEP  Estimators  Which  Have  AE  Values  Within  0.02*CEP  of  the  MAE, 
Grubbs,  Numerical,  and  7MCSV  Excluded 
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Figure  5.12  CEP  Estimators  Which  Do  Not  Have  AE  Values  Within  0.05  *CEP  of  the 
MAE,  Grubbs,  Numerical,  and  ZMCBV  Excluded 


Objective  2;  Our  second  objective  was  to  compare  similarities/diflferences  in  our  sample 
analysis  set  and  our  design  point  results. 

To  accomplish  this  objective,  we  display  grids  for  both  the  design  point  and  sample 
analysis  set  results  which  list  the  estimators  which  can  be  expected  to  deliver  an  estimate 
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of  within  0.01  *CEP  of  the  MEA: 


CEP  Estimators  With  an  AE  of  Within  0.01*CEP  of  the  Minimum  AE, 
^ _ Based  on  Our  Sample  Analysis  Set  Results  _ 
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CEP  Estimators  With  an  AE  of  Within  0.01*CEP  of  the  Minimum  AE, 
_ Based  on  Our  Design  Point  Results _ _ 
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Figure  5.13  Grids  Summarizing  Our  Sample  Analysis  Set  and  Design  Point  Results 
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Rayleigh,  Grubbs,  Numerical,  and  TMCBN  appear  the  most  robust  in  the  sense 
that  they  all  appear  in  the  majority  of  the  grid  squares  in  Figure  5.13.  The  CEP 
estimators  which  are  in  both  grids  for  similar  corresponding  cases  are  shaded;  in  a  case  by 
case  comparison  between  the  two  grids,  it  is  clear  that  the  results  are  almost  identical. 
Objective  3:  Our  third  objective  was  to  compare  our  results  with  those  of  earlier  studies. 
As  alluded  to  in  Chapter  4,  our  results  were  similar  to  those  of  Elder  (1985)  and  Tongue 
(1993),  while  different  than  those  of  Puhek  (1992).  As  mentioned  in  Section  4.2,  we 
believe  that  the  primary  reason  for  this  difference  is  the  fact  that  our  design  point  estimates 
were  an  average  of  ten  runs,  while  Puhek  did  not  conduct  replications  at  his  design  points. 
Using  single  replications  would  mean  greater  variance  and  less  accuracy  in  Puhek's  results. 
The  fact  that  our  results  did  closely  resemble  those  of  the  independent  studies  of  Elder  and 
Tongue  contributes  to  the  verification  of  our  results. 

5.2  Recommendations  for  Further  Study 

We  conclude  our  paper  with  specific  recommendations  based  on  our  results  for 
further  areas  of  study.  Tongue  asserted  that  the  future  expense  of  new  ballistic  weapon 
systems  will  make  extensive  testing  prohibitive,  resulting  in  accuracy  analysts  trying  to 
accurately  predict  CEP  with  smaller  and  smaller  sets  of  data  [1993,  page  7-8].  Tongue 
believed  that  due  to  this  expense,  further  research  was  required  in  two  areas: 

1.  Developing  new  CEP  estimators  with  greater  accuracy  for  small  samples. 

2.  Establishing  a  more  detailed  decision  grid  for  use  in  selecting  the  proper  CEP  estimator 
for  a  given  set  of  circumstances. 
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Our  recommendations  for  additional  study  focus  on  these  two  points; 

1.  Developing  Modified  Estimators'.  Tongue  [1993,  page  7-6]  recommended  that 
improved  modifications  of  Rayleigh,  Grubhs,  and  MRand  be  developed.  Tongue  based 
this  recommendation  on  the  strong  performance  of  TMCBN,  his  modification  of  the  CBN 
(Numerical)  CEP  estimator,  in  his  thesis  simulation  experiment. 

We  also  independently  verified  that  TMCBN  generally  outperformed  the  CBN 
(Numerical)  estimator: 


MSRE  Difference  Between  Numerical  and  the  Minimum  of  (Numerical,  TMCBN) 


Figure  5.14  Magnitude  of  the  MSRE  Difference  Between  Numerical,  TMCBN  ,and  the 
Minimum  MSRE  of  (Numerical,  TMCBN) 
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The  bar  graphs  in  Figure  5.14  indicate  that  the  estimation  difference  between 
Numerical  and  2MCSi\r  tends  to  increase  as  sample  size  decreases,  with  generally 

achieving  more  accurate  results.  Bear  in  mind  that  if  we  disregard  variance  as  explained 
in  Section  3.7,  an  MSRE  difference  of  0.0025  (marked  with  a  dashed  line  in  the  bar  graphs 
of  Figure  5.14)  for  a  particular  sample  analysis  set  translates  to  a  five  percent  difference  in 
the  MRE  for  a  given  set.  The  improvement  displayed  in  these  bar  graphs  appears 

to  be  substantial  for  many  sample  size  nine,  six  and  three  cases.  Based  on  these  results, 
we  join  Tongue  in  urging  that  modified  versions  of  Rayleigh,  Grubbs,  and  MRand  be 
empirically  developed  similar  to  TMCBN. 

Tongue  derived  his  TMCBN  formula  from  a  simulation  design  in  which  the 
maximum  bias  considered  was  2.0cj.  In  Figure  5.14  the  sample  analysis  sets  labeled  80- 
100  correspond  to  those  with  a  sample  bias  of  greater  than  2.75  o ;  note  that  for  these  sets 
TMCBN  usually  has  a  higher  MSRE  than  Numerical.  Like  MRand,  it  appears  that 
TMCBN  has  exclusionary  bounds  based  on  the  region  considered  during  its  development. 
In  addition  to  developing  new  modifications  for  Rayleigh,  Grubbs,  and  MRand,  Tongue's 
formula  for  modifying  the  CBN  {Numerical)  estimator  could  also  be  adjusted  for  use  at 
higher  bias  levels. 

2.  Developing  An  Improved  Decision  Grid.  Due  to  time  constraints,  we  limited  the 
number  of  levels  for  each  factor  to  five  or  less.  We  recommend  expanding  the  number  of 
factor  levels  for  the  two  most  significant  factors,  bias  and  sample  size,  in  a  future  study. 
Specifically,  we  feel  that  all  sample  sizes  between  three  and  ten  and  bias  levels  which  are 
greater  than  4.0a  should  be  explored.  With  the  additional  detail  of  such  a  design,  such  a 
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study  would  produce  more  detailed  sample  size/sample  bias  grids  than  those  produced  in 
our  study. 

Our  use  of  ten  replications  was  a  significant  improvement  over  the  earlier  CEP 
estimator  comparison  studies  which  used  no  replications.  We  recommend  that  future 
studies  use  even  more  replications  to  achieve  increased  confidence  in  the  accuracy  of  the 
results. 

As  explained  in  Section  4.1,  om  Numerical  and  ZMCBW  results  were  distorted  at 
sample  size  three.  This  distortion  was  due  to  certain  design  points  and  sample  analysis 
sets  using  differing  precision  tolerances  to  enable  the  numerical  root  finding  algorithm  to 
converge.  Perhaps  one  of  these  numerical  CEP  estimators  is  the  estimator  of  choice  at 
sample  size  three  whenever  the  root  finding  algorithm  converges  at  tolerance  0.01. 
Further  study  is  required  to  determine  the  true  performance  of  these  two  numerical 
estimators  at  this  sample  size. 

Finally,  we  close  by  recommending  that  MRand  be  reevaluated  in  a  study  that 
excludes  any  design  points  outside  MRand's  allowable  boundary  conditions.  Because  we 
included  design  points  that  were  outside  MRand 's  feasible  region,  we  could  not  fairly 
evaluate  this  estimator. 
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Appendix  A:  Notation  Used  in  the  Thesis 


A.l)  Population  Parameters  and  Sample  Statistics:  This  first  section  describes  the 
notation  used  in  this  thesis  for  the  population  parameters  and  their  corresponding  sample 
statistics.  The  formulas  for  calculating  the  sample  statistics  is  also  presented, 
n  =  The  number  of  sample  point  values. 

Px  =  The  population  crossrange  mean. 

X  =  The  sample  crossrange  mean  =  -  ^  x; .  (A-1) 

^  i=l 

Py  =  The  population  downrange  mean. 

1  " 

V  =  The  sample  downrange  mean  =  —  Y"  Yi  -  (A-2) 

*'  n  “ 


Ox  =  The  population  crossrange  standard  deviation. 


Sx  =  The  sample  crossrange  standard  deviation  =  |  ‘-i  — - 


-\2 


(A-3) 


Oy  =  The  population  downrange  standard  deviation. 


E{yi-y) 

Sy  =  The  sample  downrange  standard  deviation  =  — - 


^\2 


(A-4) 


p  =  The  population  correlation  between  corresponding  crossrange  and  downrange 
values. 


Z(x-x)(y-y) 


p  =  The  sample  correlation  - 


i=l 


(n-l)Sx  Sy 


(A-5) 


bias  =  The  population  bias  =  ^  +  Py 


(A-6) 


100 


(A-7) 


bias  =  The  sample  bias  =  +  y  ^  ■ 

r;  =  The  radial  miss  distance  for  point  (xi,  yO  =  Yi  ^  • 

f  =  The  mean  radial  miss  distance  =-y'  i; . 

ellip  =  The  sample  ellipticity  =  (  Sy/  Sx  if  Sx  >  Sy ,  Sx/  Sy  if  Sx  <  Sy  }. 

Scaling  factor  c  = 


(A-8) 

(A-9) 

(A-10) 

(A-11) 


(T  =  A  sample  estimator  for  o  = 


Sx^+S/ 


(A-12) 


In  the  above  sample  statistics,  x ,  y ,  Sx  ,  Sy  ,  and  p  are  sufficient,  minimum 
variance,  and  unbiased  estimators.  In  addition,  x  and  y  are  maximum  likelihood 


estimators  [Graybill  (1976),  pages  343-351], 

A.  2)  The  Thesis  CEP  Estimators:  In  this  section  the  notation  for  each  of  the  eight  CEP 
estimators  compared  in  the  thesis  experiment  is  presented. 

Smed  =  sample  median  CEP  estimator. 

Ethridge  =  Ethridge  CEP  estimator. 

MRand  =  modified  Rand  R-234  CEP  estimator. 

Valstar  =  Valstar  CEP  estimator. 

Grubbs  =  Grubbs-Patniak  /  chi-square  CEP  estimator. 

Rayleigh  =  Rayleigh  distribution  CEP  estimator. 

Numerical  =  direct  numerical  integration  CEP  estimator. 

TMCBN  =  Tongue's  modified  CBN  CEP  estimator. 
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A.3J  Measures  of  Effectiveness:  In  this  final  section  of  Appendix  A,  the  notation  for  the 
measures  of  effectiveness  (MOEs)  used  in  this  thesis  is  described. 


RE  =  Relative  error  =  |  actual  value  -  estimate  |  /  actual  value. 

MRE  =  The  mean  relative  error  for  a  set  of  n  estimates  =  —  VrE;  . 

var  (RE)  =  The  variance  in  the  relative  error  of  a  set  of  n  estimates 
=  — i-y'(RE:  -MRE)^  . 

MSRE  =  The  mean  square  of  the  relative  error  for  a  set  of  n  estimates 


(A-13) 

(A-14) 

(A-15) 


Appendix  B:  The  MathCAD  Design  Point  Template 

Modified  versions  of  this  template  were  used  to  find  parameters 
for  the  design  points  so  that  each  design  point  would  have  a  CEP  of 
100.  In  the  example  displayed,  the  design  points  for  bias  =  a  along 

an  axis  30®  above  the  X-axis  are  found. 


ORIGIN  :=1 


i  :=1..25 


TOL-.Ol 


1+  a 


Ux(ctx)  :=- 


sec(9) 


ay(CTx)  :=n.  ^ 


Uy(crx)  :=ux(ox)  tan(0) 


p.  :=n,  , 
M  1,1 


1.)  First,  an  initial  guess  for  ax 

is  made  using  the  Grubbs-Patniak  /  Wilson-Hilferty  formula: 

m(ox)  :=[(px(ax))^+  (py(ax))^+ (ax)^+  (oy(ax))^] 


-.8  .2 
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-.8  1.667 
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2  2  2  2 
v2(ax)  :=(px(ax))  ox  +  2-px(ax)  py(ax)  p.  ax  (oy(ax))  + (pyCox))  (oyCox)) 

v(ax)  :=2-  ex'*-!- 2-  ^p.)^-(ax)^-(oy(ax))^  -t- (ay(ax))'*  +4  v2(ax) 


.8  5 


g(ax);=  m(ax)-  1- 


9-m(ax)‘ 


guess.  for  orxe  0,0.1..  10000 

break  if  |(g(ax)  -  lOO)]  <1 


I  CPC  otherwise 
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2.)  Next,  numerical  approximation  is  used  to  find  the  approximation: 


v|/(ax,x,y)  := 


2- 


1 

(x-\ix(ax)\^ 

(x-  px(ax))-(y-  py(ax)) 

{'-Wl 

[1  -  ] 

ax(cyy(ax)) 

^  /y-  uy(CTx)\^ 
\  oy(0x)  / 


f(CTX,x,y)  :=  2-7C  CTX-(ay(ax)) 


M;(ox,x,y) 


OX.  :=  guess. 


c(i,ax)  :=root 


-100 

J 10000 -y^ 

f(CTX,x,y)  dxcfy-  .5,ctx 

m 

-100  . 

-JlOOOO-y^ 

sigx.  :=c(i,ax.) 


3.)  Last,  we  ensure  that  our  approximated  values  result  in  a  CEP  of  100: 


vi/(x,y)  :=- 


nx(sigx.)\ 


SlgXj  j 


2.p. 


[x-  px(sigx.) 

I 

[y-  py(sigXj)) 

sigXj-  ( 

cyy 

(sigx.)' 

/y-  uy(sigxj)^ 


CTy(sigx.) 


f(x,y)  := 


2-7c-sigx.* 


v(x,y) 


r.  :=  100 


r  1  ^ 

•r 

2  2 

Vr  -y 

pr(i,r)  :=root 

f(x,y)dxcfy-  .5,r 

• 

-r  • 

/  2  2 

-Vr-y  J 

CEP.  :=pr(i,r.) 


a. 

1 


(sigxjV  (n.  3-sigxj^ 


bsigx.- 


ux. 

1 


sec(0) 


uy.  :=)ix^sigxj -1311(0) 
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input,  j  :=p.  Column  1  of  the  "input"  matrix  contains  the  correlation  values 

input.  2  Column  2  of  the  "input"  matrix  contains  the  raw  unsealed  Px  values 

input.  3  =uy.  Column  3  of  the  "input"  matrix  contains  the  raw  unsealed  Py  values 

input.  ^  :  =  sigx.  Column  4  of  the  "input"  matrix  contains  the  raw  unsealed  values 

input,  j  :=n.  2  sigXj  Column  5  of  the  "input"  matrix  contains  the  raw  unsealed  Oy  values 

Finally,  the  CEP  vector  shows  the  numerically  integrated  CEP  calculation  for  each  of 
the  sets  of  {px,  py,  ax,  oy,  p}  found  in  steps  1  and  2.  Note  how  close  each  comes  to  the 
predetermined  CEP  value  of  100. 
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Appendix  C:  The  Design  Points  for  Each  Sample  Size 
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124  60  0.5  0.866  0.727  1.213  0.352  0.609  0.512  0.853  -0.8  1.667  1  0.703  .71101  1.406 

125  60  0.5  0.866  0.277  1.387  0.389  0.674  0.216  1.079  -0.8  5  1  0.778  .74978  1.334 

126  60  0.5  0.866  1.387  0.277  0.358  0.62  0.993  0.199  -0.4  0.2  1  0.716  .697531 
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151  0  2  0  1.387  0.277  0.988  0  0.685  0.137  0.4  0.2  2  0.988  .48136  2.077 

152  0  2  0  1.213  0.728  0.942  0  0.571  0.343  0.4  0.6  2  0.942  .52053  1.921 

153  0  2  0  1  1  0.898  0  0.449  0.449  0.4  1  2  0.898  .514671  1.943 


Reference  0  |ix  |Xy  Ox  Oy  ^ix  fiy  cjx  Oy  p  Oy/cx  bias  bias  CEPmpi  CEP/CEPi 
Number _ (*a)  (*a)  (*a)  (*a)  (*CEP)  (*CEP)  (*CEP)  (*CEP) _ (*a)  (*CEP)  (*CEP)  , 

154  0  2  0  0.727  1.213  0.869  0  0.316  0.527  0.4  1.667  2  0.869  .48071  2.08 

155  0  2  0  0.277  1.387  0.875  0  0.121  0.607  0.4  5  2  0.875  .42697  2.342 
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177  30  1.732  1  1.213  0.728  0.819  0.473  0.574  0.344  0.4  0.6  2  0.946  .52269  1.913 

178  30  1.732  1  1  1  0.8  0.462  0.462  0.462  0.4  1  2  0.924  .52966  1.888 

179  30  1.732  1  0.727  1.213  0.792  0.457  0.332  0.554  0.4  1.667  2  0.914  .50535  1.979 

180  30  1.732  1  0.277  1.387  0.81  0.468  0.13  0.649  0.4  5  2  1  0.935 1 .4561 8 1  2.192 


F^©T©r©nC©  0  |J,jj  jLly  CTx  M'X  M'y  ^y  P  ^y/^x  bl©S  IdISS  CEPf\/IPI  CEP/CEPmp 

Number _ (*ct)  (*q)  (*a)  (*a)  (*CEP)  (*CEP)  (*CEP)  (*CEP) _ (*a)  (*CEP)  (*CEP)  , 

181  30  1.732  1  1.387  0.277  0.857  0.495  0.686  0.137  0.8  0,2  2  0.99  .47584  2.102 

182  30  1.732  1  1.213  0.728  0.847  0.489  0.593  0.356  0.8  0.6  |  2  0.979  .49417  2.024 
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205  60  1  1.732  0.277  1.387  0.491  0.851  0.136  0.681  0.4  5  2  0.983  .4792  2.087 

206  60  1  1.732  1.387  0.277  0.479  0.83  0.664  0.133  0.8  0.2  2  0.958  .46081  2.17 

207  60  1  1.732  1.213  0.728  0.482  0.834  0.584  0.35  0.8  0.6  2  0.963  .48641  2.056 


Reference  0  i^x  fiy  Ox  Cy  i^x  jj.y  Ox  Oy  p  Oy/Ox  bias  bias  CEPmpi  CEP/CEP 
Number  (*o)  (*o)  (*a)  (*c)  (*CEP)  (*CEP)  (*CEP)  (*CEP)  (*a)  (*CEP)  (*CEP)  , 

208  60  1  1.732  1  1  0.485  0.84  0.485  0.485  0.8  1  2  0.97  .49996  2 

209  I  60  I  1  1 1.73210.727  1.213  0.489  0.847  0.356  0.593  0.8  1 1.6671  2  |  0.979  1  .4947  |  2.021 
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232  30  3.464  2  1.213  0.728  0.846  0.489  0.296  0.178  -0.4  0.6  4  0.977  .26995  3.704 

233  30  3.464  2  1  1  0.833  0.481  0.241  0.241  -0.4  1  4  0.962  .27571  3.627 

234  30  3.464  2  0.727  1.213  0.83  0.479  0.174  0.291  -0.4  1.667  4  0.959  .26498  3.774 
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259  60  2  3.464  0.727  1.213  0.489  0.846  0.178  0.296  -0.4  1.667  4  0.977  .27017  3.701 

260  60  2  3.464  0.277  1.387  0.498  0.863  0.069  0.345  -0.4  5  4  0.996  .24291  4.117 

261  60  2  3.464  1.387  0.277  0.495  0.857  0.343  0.069  0  0.2  4  0.989  .24205  4.131 
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Appendix  D;  The  Sample  Analysis  Sets  For  Each  Sample  Size 


In  this  appendix,  the  reference  number  for  each  sample  analysis  set  for  a  given 
sample  size  is  hsted. 


NOTATION  KEY 

Sx  =  sample  crossrange  standard  deviation  Sy  =  sample  downrange  standard  deviation 

2 

p  =  sample  correlation 


s.  +s„ 


a  — 


Sample  Analysis  Set 
Reference  Number 

Bias  Range 

p  Range 

Sy/Sx  Range 

1 

rO,  0.755  ] 

(-0.6,  -1.0) 

(<0.4) 

2 

I^HEISESSIHI 

(-0.6,  -1.0) 

3 

(-0.6,  -1.0) 

[0.8,  1.25] 

4 

rO,  0.755  ] 

(-0.6,  -1.0) 

(1.25,  2.5] 

5 

(-0.6,  -1.0) 

(  >  2.5) 

6 

7 

rO,  0.755  ] 

(-0.2,  -0.6] 

8 

[0,  0.755  ] 

(-0.2,  -0.6] 

[0.8,  1.25] 

9 

rO,  0.755  ] 

(-0.2,  -0.6] 

(1.25,  2.5] 

10 

(>2.5) 

11 

rO,  0.755  ] 

[-0.2,  0.2] 

HUHEuElfllH 

12 

[0,  0.755  ] 

13 

[0,  0.755  ] 

[-0.2,  0.2] 

[0.8,  1.25] 

14 

rO,  0.755  ] 

15 

[-0.2,  0.2] 

(>2.5) 

16 

[0,  0.755  ] 

(<0.4) 

17 

(0.2,  0.6] 

[0.4,  0.8) 

18 

[0,  0.755  ] 

(0.2,  0.6] 

[0.8,  1.25] 

19 

(0.2,  0.6] 

20 

rO,  0.755  ] 

(0.2,  0.6] 

(>2.5) 

21 

[0,0.755  ] 

(0.6,  1.0] 

(<0.4) 

22 

[0,  0.755  ] 

(0.6,  1.0] 

23 

[0,  0.755  ] 

(0.6,  1.0] 

[0.8,  1.25] 

24 

[0,  0.755  ] 

(0.6,  1.0] 

(1.25,  2.5] 

25 

[0,  0.755  ] 

(0.6,  1.0] 

(>2.5) 

26 

(0.755,  1.255] 

(-0.6,  -1.0) 

(<0.4) 

27 

(0.755,  1.255] 

(-0.6,  -1.0) 

[0.4,  0.8) 

28 

(0.755,  1.255] 

[0.8,  1.25] 
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Sample  Analysis  Set 
Reference  Number 


29 


30 


31 


32 


33 


34 


35 


36 


37 


38 


39 


40 


41 


Bias  Range 


p  Range 


Sy/Sx  Range 


(0.75  a,  1.25ct] 


(0.75  a,  1.255] 


(0.75  a,  1.25  a] 


(0.75H,  1.25a] 


(0.75  a,  1.25  a] 


(0.755,  1.25  a] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.755,  1.255] 


(-0.2,  -0.61 


(-0.2,  -0.6] 


(-0.2,  -0.6] 


(-0.2,  -0.6] 


r-0.2,  0.2] 


(0.2,  0.6] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.755,  1.255] 


(0.2,  0.6] 


(0.755, 

1.255] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

2.755] 

(1.255, 

,2.755] 

(1.255, 

,2.755] 

(1.255, 

,2.755] 

(1.255, 

,2.755] 

(1.255, 

,2.755] 

(1.255, 

,2.755] 

(0-6,  1.0] 


(0-6,  1.0] 


(0-6,  1.0] 


(06,  1.0] 


(-0.6,  -1.0) 


(-0.6,  -1.0) 


(-0-6,  -1.0) 


(-0.2,  -0.6] 


(1.255,2.755] 


(1.255,2.755] 


(1.255,2.755] 


[-0.2,  0.2] 


(0.2,  0.6] 


[0.4,  0.8) 


[0.8,  1.25] 


(1-25,2.5] 


(>2.5) 


[0.8,  1.25] 


(  >  2.5) 


[0.8,  1.25] 


(>2.5) 


(<0-4) 


[0.8,  1.25] 


(1.25,  2.5] 


(>2.5) 


[0.8,  1.25] 


(1-25,  2.5] 


(>2.5) 


(<0.4) 


(<0.4) 


[0.4,  0.8) 
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Bias  Range 


p  Range 


Sy/Sx  Range 


Sample  Analysis  Set 
Reference  Number 


68 


69 


(1.25a,  2.75a] 


(1.25a,  2.75a] 


(1.25a,  2.75a] 


(1.25a,  2.75a] 


(1.25a,  2.75a] 


(1.25a,  2.75a] 


(>  2.75  a  ) 


(>2.75  a  ) 


(>2.75  a  ) 


(0.2,  0.6] 


(-0.6,  -1.0) 


(-0-6,  -1.0) 


[-0.2,  0.2] 


[-0.2,  0.2] 


(0.2,  0.6] 


(0.2,  0.6] 


(0.2,  0.6] 


(0.6,  1.0] 


(0.6,  1.0] 


(1.25,  2.5] 


(>2.5) 


(<0.4) 


[0.4,  0.8) 


[0.8,  1.25] 


(1.25,  2.5] 


[0.8,  1.25] 


(1.25,  2.5] 


(>2.5) 


(<0.4) 


(-0.2,  -0.6] 

[0.8,  1.25] 

(-0.2,  -0.6] 

(1.25,  2.5] 

(-0.2,  -0.6] 

(>2.5) 

[0.8,  1.25] 


(1.25,  2.5] 


(>2.5) 


(<0.4) 


(>2.5) 


(<0.4) 


[0.4,  0.8) 


98 

(>2.75  a  ) 

(0.6,  1.0] 

[0.8,  1.25] 

99 

(>2.75  a  ) 

(0.6,  1.0] 

(1.25,  2.5] 

100 

(>2.75  a  ) 

(0.6,  1.0] 

(>2.5) 
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Appendix  E;  The  MODSM  Sample  Generator  Program 

For  an  input  sample  size  n  and  px,  l-iy,  Ox,  and  p  values  from  an  input  data  file, 
this  program  was  used  to  create  n  simulated  (x,  y)  coordinates.  The  program  generates 
sample  statistics  and  other  data  required  for  input  into  the  MathCAD  CEP  estimator 
template.  The  program  as  displayed  was  used  to  generate  the  sample  output  found  in 
Table  3.4  on  page  31. 


MAIN  MODULE  samp ; 

FROM  lOMod  IMPORT  StreamObj,  ALL  FileUseType  ; 

FROM  RandMod  IMPORT  RandomObj ; 

FROM  MathMod  IMPORT  EXP,  LN,  SIN,  COS,  POWER,ATAN,  pi; 
VAR 

sin,  sout :  StreamObj ; 

ux,  uy,  ox,  oy,  p,  x,  y,  theta,  xsum,  ysum,  rbar,  biasum,  ot2, 
uxtrans,  uytrans,  oxtrans,  oytrans,  xtrans,  ytrans,  tsum,  tbar, 
xbar,  ybar,  sx,  sy,  pbar,  nreal,  sxsum,  sysum,  psum,  b,  stsum, 
knum,  kden,  ks,  med,  sd,  smedian,  u,  d,  t,  dsum,  w,  ethridge 
;REAL; 

n,  k,  h,  i,  j,  q,  rep,  input,  stop  :  INTEGER; 
median  ;  ARRAY  INTEGER  OF  REAL  ; 
space  :  STRING ; 

(  In  MODSIM,  subroutines  are  called  "procedures"  and  must 
be  written  PRIOR  to  the  main  body) 

PROCEDURE  compute(IN  ux,uy,ox,oy,p:REAL; 

OUT  a,uxt,uyt,oxt,oyt:REAL) ; 

VAR 

square  1,  square2  :  REAL ; 

BEGIN  (  This  procedure  finds  the  transformed  values  of 
ox,  oy,  ux,  and  uy  } 

IF  ox  o  oy 

a  :=  0.5*(ATAN((2.0*p*ox*oy)/(ox*ox-oy*oy))) ; 

ELSIF  p  >  0.0 
a  :=  -1.0*pi/4.0 ; 
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ELSIF  p  <  0.0 
a  :=  pi/4.0 ; 

ELSIF  p  =  0.0 
a  ;=  0.0 ; 

END  IF; 

uxt  :=  ux*COS(a)  +  uy*SIN(a) ; 
uyt  :=  -1.0*ux*SIN(a)  +  uy*COS(a) ; 

squarel:=(ox*ox-oy*oy)*(ox*ox-oy*oy)+4.0*p*p*ox*ox*oy*oy; 

square2  ;=  POWER(squarel,  0.5); 

IF  ox  >  oy 

oxt  ;=  POWER(((ox*ox  +  oy*oy  +  square2)/2.0),0.5); 
oyt  :=  POWER(((ox*ox  +  oy*oy  -  square2)/2.0),0.5); 

ELSE 

oyt  :=  POWER(((ox*ox  +  oy*oy  +  square2)/2.0),0.5); 
oxt  ;=  POWER(((ox*ox  +  oy*oy  -  square2)/2.0),0.5); 

END  IF; 

END  PROCEDURE ; 

PROCEDURE  select  (IN  uxt,uyt,oxt,oyt:REAL;  IN  k:INTEGER; 

OUT  xtrans,ytrans:REAL) ; 

VAR 

rl,r2  :  RandomObj ; 
k2  :  INTEGER ; 

BEGIN  (This  procedure  selects  a  random  x  and  y  value  from  } 

NEW(rl)  ;  (the  bivariate  normal  distribution} 

NEW(r2) ; 
k2:=  k*10; 

ASK  rl  TO  SetSeed(k)  ; 

ASKr2TO  SetSeed(k2); 

xtrans  ASK  rl  TO  Normal  (uxt,  oxt)  ; 

ytrans  ;=  ASK  r2  TO  Normal  (uyt,  oyt) ; 

DISPOSE(rl) ; 

DISPOSE(r2) ; 

END  PROCEDURE ; 

BEGIN  (main  body} 

input  :=  35  ;  {  You  must  manually  enter  the  number  of  data  sets  in 

the  input  file  name  in  the  following  statement  } 

n  :=  1000  ;  {  You  must  manually  enter  the  desired  sample  size  n  for 

each  sample  set  created  in  the  following  statement  } 
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OUTPUT("  n  pbar  xbar  ybar  sx  sy 
"smedian  rbar  ethridge"); 


OUTPUT ; 


nreal  ;=  FLOAT(n) ; 


k  :=  1 ; 
xsum  :=  0.0 ; 
ysum  :=  0.0 ; 
sxsum  ;=  0.0 ; 
sysum  :=  0.0 ; 
psum  ;=  0.0 ; 
sd  ;=  0.0 ; 


{  k  =  seed  position,  must  be  manually  entered.} 

{  For  n  =  3,  set  k  =  1  } 

{  For  n  =  6,  set  k  =  50000  } 

{  For  n  =  9,  set  k  =  100000  } 

{  For  n  =  15,  set  k  =  150000  } 


biasum  :=  0.0 ; 
knum  :=  0.0  ; 
stsum  :=  0.0 ; 
dsum  :=  0.0  ; 


u  ;=  0.0 ; 
tsum  :=  0.0 ; 
smedian  :=  0.0 ; 
NEW(sin)  ; 
space  "  " ; 


{You  must  manually  enter  the  input  file  name  into  the 
following  statement } 


ASK  sin  TO  Open  (''d.txt",Input)  ; 

{  The  input  file  name  is  entered  inside  the  italices  ("  ”)  } 


FOR  h  ;=  1  TO  input 

ASK  sin  TO  Readlnt(rep)  ; 

ASK  sin  TO  ReadReal(ux)  ; 

ASK  sin  TO  ReadReal(uy) ; 

ASK  sin  TO  ReadReal(ox) ; 

ASK  sin  TO  ReadReal(oy) ; 

ASK  sin  TO  ReadReal(p)  ; 

FORi:=l  TOrep 
NEW  (median,  l..n) ; 

compute(ux,uy,ox,oy,p,theta,uxtrans,uytrans,oxtrans,oytrans); 
OUTPUTC Analysis  sample  #",h,"  rep  #  ",i," ;")  ; 

OUTPUT ; 

FORj  ;=  1  TOn 
k  :=  k+1 ; 

select(uxtrans,uytrans,oxtrans,oytrans,k,xtrans,ytrans) ; 

X  ;=xtrans*COS(theta)-ytrans*SIN(theta) ; 
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y  :=xtrans*SIN(theta)+ytrans*COS(theta) ; 
xsum  :=  xsum  +  x ; 
ysum  ;=  ysum  +  y  ; 
b  :=  POWER((x*x+y*y),0.5); 
median[j]  :=  b ; 
biasum  :=  biasum  +  b  ; 
tsum  :=  tsum  +  LN(b)  ; 

{  OUTPUTC'x  =  ",x,  space, "y  =  '',y) ; } 

END  FOR ; 
xbar  :=  xsum/nreal ; 
ybar  ;=  ysum/nreal ; 
tbar  :=  tsum/nreal ; 
rbar  :=  biasum/nreal ; 
k  ;=  k-n ; 

{  routine  for  finding  the  sample  median  } 

IF  ODD(n)  =  TRUE 

stop  :=  TRUNC(nreal/2.0) ; 

FORj  :=  1  TO  stop 
med  :=  0.0 ; 

FOR  q  :=  1  TO  n 

med  :=  MAXOF(median[q],med); 

END  FOR ; 

FOR  q  :=  1  TO  n 
IF  median[q]  =  med 
median  [q]  :=  0.0  ; 

END  IF; 

END  FOR ; 

j ”j+i ; 

END  FOR ; 
smedian  ;=  0.0 ; 

FORq  :=  1  TOn 

smedian  :=  MAXOF(median[q],smedian) ; 
END  FOR ; 

ELSE 

stop  :=  TRUNC(nreal/2.0) ; 

FORj  ;=  1  TO  stop 
med  :=  0.0  ; 

FOR  q  ;=  1  TO  n 

med  ;=  MAXOF(median[q],med); 

END  FOR ; 

FOR  q  :=  1  TO  n 
IF  median[q]  =  med 
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median  [q]  ;=  0.0  ; 

END  IF; 

END  FOR ; 

j  :=j  + 1 ; 

END  FOR ; 
smedian  ;=  0.0 ; 

FORq;=lTOn 

smedian  :=  MAXOF(median[q], smedian)  ; 

END  FOR ; 

smedian  ;=  (smedian  +  med)/2.0  ; 

END  IF; 

FOR  j  :=  1  TO  n 
k  :=  k+1 ; 

select(uxtrans,uytrans,oxtrans,oytrans,k,xtrans,5drans) ; 

X  ;=  xtrans*COS(theta)-ytrans*SIN(theta) ; 

y  ;=  xtrans*SIN(theta)+ytrans*COS(theta) ; 
b  :=POWER((x*x+y*y),0.5); 
t;=LN(b); 

sxsum  :=  sxsum  +  POWER((x  -  xbar),2.0)  ; 
sysum  :=  sysum  +  POWER((y  -  ybar),2.0)  ; 
psum  ;=  psum  +  (x-xbar)*(y-ybar) ; 
knum  ;=  knum  +  POWER(t-tbar,4.0); 
stsum  :=  stsum  +  POWER(t-tbar,2.0); 

END  FOR ; 

sx  ;=  POWER((sxsuni/(nreal-1.0)),0.5) ; 
sy  :=  POWER((sysum/(nreal-1.0)),0.5) ; 
pbar  :=  psum/((nreal-1.0)*sx*sy) ; 
ot2  :=  stsum/(nreal  - 1.0)  ; 
kden  :=  POWER(stsum,  2.0)  ; 
ks  ;=  knum/kden ; 
k  k-n ; 

FORj:=lTOn 
k  :=  k+1 ; 

select(uxtrans,uytrans,oxtrans,oytrans,k,xtrans,ytrans) ; 

X  ;=xtrans*COS(theta)-ytrans*SIN(theta) ; 
y  :=  xtrans*SIN(theta)+ytrans*COS(theta) ; 
b  :=POWER((x*x+y*y),0.5); 
t:=LN(b); 
d:=1.0- 

(0.03*(POWER((ks-3.0),3.0))*(t-smedian)*(t-smedian))/ot2; 
IF  d  <=  0.01 
d:=0.01  ; 

END  IF; 
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dsum  dsum  +  (1.0/d) ; 

END  FOR; 
k  :=  k-n ; 

FOR  j  :=  1  TO  n 
k  :=  k+1  ; 

select(uxtrans,uytrans,oxtrans,oytrans,k,xtrans,ytrans) ; 

X  :=  xtrans*COS(theta)-ytrans*SIN(theta) ; 
y  :=xtrans*SIN(theta)+ytrans*COS(theta) ; 
b  :=POWER((x*x+y*y),0.5); 
t:=LN(b); 
d  :=  1.0- 

(0.03*(POWER((ks-3.0),3.0))*(t-smedian)*(t-smedian))/ot2; 
IF  d  <=  0.01 
d;=0.01  ; 

END  IF; 

w  :=  (1.0/d)  /  dsum ; 
u  :=  u  +  w*t ; 

END  FOR ; 
ethridge  ;=  EXP(u) ; 

OUTPUT(n,"  ",pbar,"  ",xbar,"  ",ybar,"  ",sx,''  ",sy," ", 
smedian,"  ",rbar," ", ethridge); 

xsum  ;=  0.0 ; 
ysum  ;=  0.0 ; 
sxsum  ;=  0.0 ; 
sysum  :=  0.0 ; 
psum  :=  0.0 ; 

DISPOSE  (median) ; 
sd  :=  0.0  ; 
biasum  :=  0.0 ; 
knum  ;=  0.0 ; 
stsum  :=  0.0 ; 
dsum  :=  0.0 ; 
u  :=  0.0 ; 
smedian  :=  0.0 ; 
tsum  :=  0.0 ; 

END  FOR ; 

END  FOR ; 

ASK  sin  TO  Close ; 

DISPOSE  (sin) ; 

END  MODULE. 
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Appendix  F;  The  MathCAD  CEP  Estimator  Template 

This  template  takes  in  output  from  the  MODSIM  sample  generator  program.  It 
outputs  the  CEP  estimate  for  the  each  of  the  8  thesis  CEP  estimators  for  each  input 
simulation  run.  In  the  example  displayed,  the  input  is  from  design  point  20  for  sample 
size  15: 


ORIGIN  :=  1 
TOL  =0.01 

input  ;  =  READPRN  (popl5) 

a  ;=191 
b  =200 
n  ;=  10 

s  =  submatrix  (input,  a,  b,  1,  9) 
i  :=  1..  n 


Input  Variable  Identification; 


n 

pbar 

xbar 

ybar 

Sx 

Sy 

Smed 

rbar  Ethridge 

15 

0.141 

47.57 

8.641 

21.065 

87.89 

93.568 

91.499 

82.542  ■ 

15 

0.085 

44.637 

26.311 

18.122 

151.461 

84.27 

130.741 

108.669 

15 

0.044 

47.788 

56.131 

18.011 

123.134 

109.932 

124.324 

104.501 

15 

-0.429 

41.111 

11.558 

31.451 

118.894 

119.437 

117.336 

105.475 

15 

-0.026 

44.979 

25.827 

24.911 

109.761 

83.48 

102.614 

88.238 

15 

0.037 

37.786 

35.591 

21.951 

148.713 

114.44 

133.202 

113.387 

15 

0.57 

45.209 

19.972 

24.782 

120.626 

108.014 

113.531 

95.339 

15 

-0.029 

50.522 

23.133 

28.195 

113.19 

107.833 

116.402 

105.595 

15 

-0.158 

35.351 

16.15 

29.818 

106.429 

89.328 

97.421 

79.691 

15 

-0.49 

44.933 

35.422 

27.796 

172.219 

125.115 

152.514 

126.34 
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Smed: 

Sample  Median  Estimator 


2 


CEP  :=s  CEP.  ,-100 

*•'  rel.  ,  := - iii - '  srel.  , 

1,1  100 


CEP.  ,  -  100 
1,1 _ 

100 


Ethridge: 

Ethridge  CEP  Estimator 


re2. 


CEP. 


i,2 


-  ioo| 


i,l 


100 


sre2. 


CEP.  --  100 

1,.4 


i,l 


100 


Grubbs:  Grubbs-Patniak  Chi-Square  CEP  Estimator 


iXs)  ih 

■  =  H(\5y+^-  h.2)'  (^6)'+  (®i,6)1  +  4-v2. 


vqchisq(.5,d) 

CEP.  - - ^ 

I'ta. 


CEP.  too 


sre5.  ,  := 

1 


CEP.  100 


Rayleigh.  ,  _  93 9^.  g 

Rayleigh  Distribution  *’* 

CEP  Estimator 


CEP.  100 


CEP.  100 


Numerical: 

The  Direct  Numerical  Integration  CEP  Estimator: 


Ci(x,y) 


1^]  _2.s.  [(^~^3)-(y~0]  /y-\4 

%5  /  ’  _  ®i,5'®i,6  .  \  %6  , 


f(x,y) 


=  the  joint  bivariate 
normal  distribution. 


r.  ;=CEP.  , 

1  1,5 


rr  Ur^-y^ 


pr(i,r)  :=root 


f(x,y)dxdy-  .5,r 


r  J-Vr^-y2 


CEP.^,  :=pr(i.r. 


CEP.  100 
/ 


CEP.  100 
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TMCBN:  Tongue's  Modified  CBN  (Correlated  Bivariate  Normal)  Estimator 

^  CORR.  :=s.  ,  BIAS.  := 

,6  1  1,2  1 

,6 

REl.  :=  .171833  -  .009784  s.  ,  -  .037707  BIAS.  -  .150628  STDR.  +  .002045  s.  -BIAS,  -i-  .007488  s.  -STDR 
•  1,1  1  1  1,1  1  1,1  1 

REl  :=.019014BIAS.  STDR.-i-.116385  STDR.-CORR.-  .006714  STDR.-CORR.  S 

1  II  11  1  1  1,1 

RE.  :=RE1.  +  RE2. 

1  1  1 


CEP  -RE' 

CEP. 

\  re8  *’* 

100 

sre8 

II 

00 

1 

»— * 
O 
O 

1,8  i  V 

'  100 

i, 

^'  \  100  / 

mrel  :  =  mean(rel) 

mre2  :=mean(re2) 

mre3  :==mean(re3) 

mre4  :=mean(re4) 

mre5  :  =  mean(re5) 

mre6  :  =  mean(re6) 

mre?  :  =  mean(re7) 

mre8  :  =  mean(re8) 

vrel  :=var(rel) 

vre2  :=var(re2) 

vre3  :=var(re3) 

vre4  :=var(re4) 

vre5  :=var(re5) 

vre6  :=var(re6) 

vre?  :=var(re?) 

vre8  :=var(re8) 

msrel  :=meaii(srel) 

msre2  :=mean(sre2) 

msre3  :=mean(sre3) 

msre4  :=mean(sre4) 

msreS  :=mean(sre5) 

msre6  :-mean(sre6) 

msre?  :=mean(sre?) 

msre8  :=meaii(sre8) 

mrel  =0.134 

mre2  =0.118 

mre3  =0.111 

mre4  =0.138 

vrel  =0.0032 

vre2  =0.0051 

vre3  =  0,0046 

vre4  =0.0059 

msrel  =0.021 

insre2  =0.019 

msre3  =0.01? 

msre4  =  0.025 

mre5  =0.109 

mre6  =0.16 

mre?  =0.104 

mre8  =0.103 

vre5  =  0.009 

vre6  =0.0125 

vre?  =0.00?? 

vre8  =  0.0065 

msreS  =0.021 

msre6  =0.038 

msre?  =0.019 

msre8  =0.01? 

mre  :-(mrel  mre2  inre3  inre4  mre5  mreO  nire7  nire8)  =  the  mean  relative  error  values 
vre  (vrel  vre2  Yre3  vre4  vre5  vre6  vre7  vre8 )  =  the  variance  from  the  MRE  values 

msre:=(msrel  msre2  insre3  nisre4  msreS  msre6  msre7  msre8)  =  the  MSRE  values 
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Output  Identification 


CEP 


msre 
mre  = 

vre  = 


Smed 

Ethridge 

MRand 

Valstar 

93.568 

82.542 

75.646 

76.713 

84.27 

108.669 

106.15 

114.71 

109.932 

104.501 

104.237 

111.19 

119.437 

105.475 

92.236 

91.564 

83.48 

88.238 

89.489 

90.595 

114.44 

113.387 

106.928 

113.119 

108.014 

95.339 

91.511 

95.858 

107.833 

105.595 

94.579 

94.643 

89.328 

79.691 

84.794 

81.948 

125.115 

126.34 

122.308 

130.003 

Grubbs  Rayleigh  Numerical  TMCBN 


82.771 

85.955 

81.902 

80.71 

116.91 

122.818 

114.744 

112.637 

108.723 

116.79 

105.845 

104.15 

98.623 

110.226 

98.134 

96.939 

95.334 

96.396 

94.431 

93.061 

114.252 

125.13 

112.713 

110.747 

98.96 

106.651 

97.213 

95.535 

101.094 

109.348 

100.49 

99.096 

88.988 

91.517 

89.043 

87.903 

132.812 

143.272 

130.782 

128.714 

Smed  Ethridge  MRand  Valstar  Grubbs  Rayleigh  Numerical  TMCBN 


=  (  0.0212  0.0191 
=  (0.1341  0.1182 
(  0.0032  0.0051 


0.017  0.0249  0.0209 
0.1114  0.1377  0.1091 
0.0046  0.0059  0.009 


0.0382  0.0185  0.0171) 
0.1604  0.1039  0.103) 
0.0125  0.0077  0.0065 


=  the  MSRE  values 
=  the  MRE  values 
)  =  the  VRE  values 
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Appendix  G;  The  Design  Point  MSRE  Results 


Sample  Size  15 


Smed 

Ethridge 

MRand 

Rayleigh 

0.0757 

■weectI 

0.0229 

0.0341 

0.0313 

0.0603 

0.03 

0.0222 

0.0118 

0.0125 

0.012 

0.0127 

0.012 

lifillEI 

0.0175 

0.0246 

0.0071 

0.0071 

0.0071 

0.0083 

0.0072 

0.0322 

0.0151 

0.0094 

0.0099 

0.0099 

0.0754 

0.0156 

0.0138 

0.0173 

0.0191 

0.0417 

0.0187 

0.0177 

0.0541 

0.0233 

0.0162 

0.0251 

0.0219 

0.0395 

0.0215 

0.0205 

0.0275 

0.011 

0.0063 

0.0106 

0.0126 

0.0102 

0.0105 

0.0395 

lififeftSl 

0.0146 

0.0149 

0.0154 

0.0147 

0.0551 

0.0372 

0.0215 

0.0208 

0.021 

0.0217 

0.063 

0.0218 

0.0195 

0.0292 

0.0226 

0.0332 

0.0226 

0.0229 

0.0936 

0.037 

0.0153 

0.0265 

0.0262 

0.059 

0.0261 

0.0245 

0.034 

0.041 

0.0114 

0.013 

0.0144 

0.027 

EOS 

0.0218 

0.0298 

0.0173 

0.0262 

0.0183 

0.0176 

0.0297 

0.0143 

0.0133 

0.0189 

0.0122 

0.0213 

0.0112 

0.011 

0.0246 

0.0218 

0.0127 

0.0121 

0.0123 

0.0121 

0.0422 

0.0421 

0.0203 

0.0278 

0.0186 

0.019 

0.0351 

0.0192 

0.0205 

0.0154 

0.0163 

0.0148 

0.0143 

0.0242 

■IlIlKfil 

0.0241 

0.018 

0.0262 

0.0164 

0.0154 

0.015 

isissm 

0.0191 

0.017 

0.0382 

0.0185 

lilillrifl 

0.0406 

0.0289 

0.0273 

0.0472 

0.027 

0.057 

0.0244 

0.0383 

0.0251 

0.0265 

0.0331 

0.0208 

0.0291 

0.0197 

0.0138 

0.0341 

0.0096 

0.009 

0.0089 

0.014 

0.0087 

0.0095 

0.0124 

0.0067 

0.0124 

0.015 

0.0114 

0.0077 

0.0111 

0.0223 

0.0083 

0.0062 

0.0067 

0.0855 

0.0317 

0.0197 

0.0502 

0.0292 

0.0282 

0.0723 

0.0384 

0.0121 

0.0273 

0.012 

0.0294 

0.0125 

0.0204 

0.0326 

0.0145 

0.0162 

0.0127 

Em 

0.0265 

0.0112 

0.0117 

0.0151 

0.0089 

0.0087 

0.0422 

0.0129 

0.0112 

0.0405 

0.0187 

0.0168 

0.0395 

0.0138 

0.0089 

0.0233 

0.0172 

0.0504 

0.0152 

0.0134 

0.0563 

0.0333 

0.0194 

0.0185 

0.0233 

0.0223 

0.0286 

0.0172 

0.0063 

0.0061 

0.0101 

0.0158 

0.0355 

0.0207 

0.01 

0.0145 

0.0106 

0.0213 

0.0097 

0.009 

0.0232 

0.0167 

0.0125 

0.0168 

0.0155 

0.034 

0.0132 

0.0122 

0.0643 

0.0197 

0.0108 

0.0171 

0.0173 

0.043 

0.0158 

0.0145 

Smed  \Ethridge\MRand 


Rayleigh  Numericall 


0.0427 

0,0122 

0,0337 

0,0407 

0,023 

0,0713 

0.0193 

0,024 

0,0222 

0,0707 

0,0487 

0,0276 

0,0441 

0,015 

0.0878 

0,0186 

0,0318 

0,0503 

0,0508 

0,0537 

0,0296 

0,0328 

0,0336 

0,0248 

0,077 

0,0224 

0,0378 

0.0163 

0,0873 

0,022 

0,0251 

0,0276 

0.0417 

0,0595 

0.0696 

0,0301 

0,0292 

0,0387 

0,0334 

0.0413 


0,0144 

0,0066 

0,0194 

0,0138 

0,0178 

0,0323 

0,0267 

0,0214 

0,0093 

0,0433 

0,0327 

0,0275 

0,0286 

0,0067 

0,0872 

0,022 

0.0213 

0.0334 

0,0204 

0.0178 

0,0164 

0,0103 

0,0249 

0,0098 

0,0284 

0,0083 

0,0319 

0,0098 

0,0113 

0.0145 

0,0299 

0,022 

0,024 

0,0296 

0,047 

0,0279 

0,0183 

0,0349 

0,0214 

0.0085 


0,016 

0,0072 

0,0188 

0,0169 

0,0099 

0,0186 

0,0184 

0,0171 

0,0138 

0.0266 

0,0191 

0,0123 

0,0279 

0,0113 

0,0501 

0.0212 

0,0265 

0,0239 

0,0172 

0,0204 

0,0147 

0,0064 

0,0145 

0,0168 

0,0227 

0,0055 

0,0142 

0,0036 

0.0101 

0,0152 

0,007 

0,0154 

0,013 

0,0327 

0,0274 

0,0095 

0.0191 

0,03 

0,0595 

0.014 


Valstar 


Grubbs 


0,025 

0,0085 

0,0215 

0,0218 

0.0177 

0,0222 

0.0177 

0,0236 

0,0159 

0.039 

0,0225 

0,0156 

0,034 

0,014 

0,0754 

0,0217 

0,0339 

0,029 

0,0136 

0,0176 

0,0243 

0,0099 

0,0152 

0.0216 

0,0257 

0,004 

0,0147 

0,0057 

0.018 

0,0144 

0,0054 

0,0169 

0.0183 

0,0396 

0.0327 

0,008 

0,0269 

0,0406 

0,0768 

0.0092 


0,0187 

0,0074 

0,0182 

0,0183 

0,0167 

0,0178 

0,0193 

0,0196 

0,0103 

0.0301 

0,0191 

0,0105 

0,0278 

0,0122 

0,0513 

0,0194 

0,0182 

0,0205 

0,0104 

0,0126 

0,0174 

0,0063 

0,016 

0,011 

0,0283 

0,0059 

0.0141 

0,0029 

0,0137 

0.0127 

0,0057 

0,0163 

0,0103 

0,0312 

0,0292 

0,0087 

0,0166 

0,0261 

0,0442 

0.0069 


0,0171 

0,0052 

0,0186 

0,0351 

0,0357 

0,028 

0,0183 

0,02 

0,0153 

0,0434 

0,0279 

0,0105 

0,0301 

0,0225 

0,0826 

0,0384 

0,0217 

0,0304 

0,0258 

0,0272 

0,0264 

0,0121 

0,02 

0.02 

0,0479 

0,0076 

0,0187 

0,0027 

0,0308 

0,0254 

0,0087 

0,015 

0,0135 

0,0499 

0,0541 

0,0128 

0,0193 

0,0277 

0,0514 

0.0158 


0,0199 

0,0077 

0.0186 

0,0157 

0.0155 

0,0183 

0,0193 

0,0206 

0,0092 

0.0287 

0,0192 

0,0104 

0,0287 

0,0115 

0,0497 

0,0175 

0.0171 

0,0191 

0,0102 

0,0127 

0.016 

0,0063 

0.0158 

0,0098 

0,0264 

0,006 

0,0143 

0,0031 

0,0124 

0,0125 

0,0053 

0,0162 

0,0103 

0,0294 

0.0286 

0,0085 

0,0166 

0,0256 

0,0406 

0.0074 


TMCBM 


0,0182 

0.007 

0,0179 

0,0141 

0.0145 

0,017 

0,0197 

0,0194 

0,0091 

0,0275 

0,0195 

0,0102 

0,027 

0,0111 

0,0481 

0,0163 

0,0161 

0,0185 

0,0104 

0,0128 

0,0152 

0,0057 

0.0153 

0,0092 

0,0251 

0,0059 

0,0142 

0,003 

0,0112 

0,0125 

0,0063 

0,016 

0,0098 

0,0282 

0,0284 

0,009 

0,0157 

0,0256 

0,0391 

0.0078 


Design  Point 
Reference  Number 


77 


78 


79 


1  80 


81 


82 


83 


373 


0.0284 


0.0487 


0.0469 


0.0359 


0.0269 


0.0932 


0.0416 


0.0214 


0.0397 


0.0361 


0.0097 


0.0178 


0.0081 


0.044 


0.0158 


0.0085 


0.012 


0.0106 


0.0393 


0.0386 


0.02 


0.0091 


0.0069 


0.0187 


0.0324 


0.021 


0.0341 


0.0246 


0.014 


Ethridge 

MRand 

0.0246 

0.011 

0.0147 

0.0265 

liiliMdl 

0.0276 

IwittnicM 

0.0273 

0.0385 

0.0431 

0.0162 

0.0219 

0.0347 

0.0082 

0.0053 

0.0108 

0.0528 

0.0306 

0.0266 

0.0247 

0.0158 

0.0144 

mm 

0.0071 

0.0386 

0.0228 

0.0146 

0.0156 

0.0186 

0.0174 

0.0169 

0.01 

0.0097 

0.018 

0.0294 

0.0148 

0.0111 

0.0057 

0.0057 

0.0055 

0.0102 

0.0094 

0.0069 

0.0259 

0.014 

0.027 

0.0133 

0.0236 

0.0101 

0.011 

0.0035 

0.0074 

0.0101 

0.0127 

0.0142 

0.0183 

0.0148 

0.0229 

0.0151 

0.0129 

0.013 

0.011 

0.0118 

0.0172 

0.0218 

0.0221 

0.0328 

0.0159 

0.0171 

0.0212 

0.0167 

0.0103 

0.0165 

0.0079 

0.0241 

0.0565 


0.0364 


0.0127 


0.007 


0.0069 


0.0438 


0.0351 


0.0452 


0.0322  0.0395 


0.0131  0.0113 


0.0077  0.0103 


Rill 

ES 

m 


92 


0.0214 


0.0224 


0.0279 


0.0122 


0.0079 


0.0057 


0.0035 


0.0244 


0.018 


0.0136 


0.0057  0.0093  0.0134 


0.0596  0.0291  0.0354 


0.0172  0.013  0.0296 


0.0136 


RJKB® 


0.0461 


0.009 


0.0063 


0.01 


0.0074 


0.0199 


0.0189 


0.0124 


0.0041 


0.0108 


0.0172 


0.0212 


0.0252  0.0352 


0.0093  0.0126 


0.0026  0.0023 


0.0059  0.0072 


0.0103  0.0141 


0.0276 


0.0113 


0.0121 


0.009 


0.007 


0.0202 


0.0224 


0.0263 


0.0105 


0.0075 


0.0053 


0.0036 


0.0242 


0.0171 


0.0125 


0.0157 


0.0053 


0.0263 


0.0109 


0.0111 


0.0082 


0.0067 


0.011  0.0113 


0.0021  0.0026 


0.0095  0.0114 


0.0125  0.0107 


QE 

M 


0.0379 


0.0423 


0.0267 


0.0203 


P 


0.0147 


0.0192  0.0408 


0.0243  0.0196 


0.0173  0.0151 


0.0024 

0.0023 

0.0055 

0.0056 

0.0051 

0.0045 

0.0105 

0.0105 

0.0149 

0.0142 

0.0115 

0.0106 

0.0022 

0.002 

0.006 

0.0054 

0.0112 

0.0114 

0.0123 

0.0119 

0.0165 

0.0153 

0.0166 

0.0157 

0.007 

0.0064 

wBBsmwmm 

0.0229 

0.0225 

0.0154 

0.0141 

0.0171 

0.0165 
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Design  Point 
Reference  Number 


117 


118 


119 


Smed  \Ethridge\MRand 


RayleighlNumericall  TMCB. 


0.0412 

0.0477 

0.0274 

0.0287 

0.0242 

0.0232 

0.0239 

0.0246 

0.0157 

0.023 

0.0391 

0.0446 

0.0258 

0.0183 

0.0234 

0.023 

0.0333 

0.0062 

0.0198 

0.0273 

0.0132 

0.0175 

0.0099 

0.0088 

0.0154 

0.0077 

0.0123 

0.0181 

0.0121 

0.0195 

0.0074 

0.0065 

0.0136 

0.0145 

0.0104 

0.0117 

0.0131 

0.0198 

0.0099 

0.0099 

0.0071 

0.0104 

0.0118 

0.0078 

0.0056 

0.0092 

0.0057 

■wimaei 

0.0145 

0.011 

0.0106 

0.0134 

0.0096 

0.0104 

0.0094 

0.0094 

0.0123 

0.0113 

0.0054 

0.0058 

0.0057 

0.0178 

■wanAi 

0.0262 

0.0402 

0.0112 

0.0098 

0.0078 

0.0106 

0.0052 

0.005 

0.0304 

0.0266 

0.016 

0.0148 

0.0133 

liftIkH 

0.0206 

0.0161 

0.007 

0.0077 

0.0079 

0.0073 

0.0084 

0.0159 

0.0282 

0.012 

0.0092 

0.01 

0.014 

0.0093 

0.0105 

0.008 

0.0345 

0.0474 

0.0241 

0.0257 

liISIEfSI 

0.0149 

0.0139 

0.0141 

0.0105 

0.017 

0.0088 

0.0089 

0.0108 

0.0158 

0.0091 

0.0088 

0.0096 

0.0105 

0.0098 

0.0098 

0.0116 

0.02 

0.0097 

0.0108 

0.0106 

0.0133 

0.0283 

0.0066 

0.0078 

0.0157 

0.0095 

0.0246 

■iittMipintga 

0.0145 

0.0144 

0.0251 

0.0106 

0.01 

0.0071 

0.0099 

0.0148 

0.0063 

0.0106 

0.0047 

0.0045 

0.0252 

0.0189 

0.0151 

0.0197 

0.0122 

0.0125 

0.0126 

0.023 

0.0117 

0.0303 

0.0427 

0.0201 

0.0173 

0.0187 

0.0178 

0.0147 

0.0107 

0.0153 

0.0226 

0.0108 

0.0141 

0.0102 

0.0105 

0.0377 

0.023 

0.0131 

0.0277 

0.0242 

0.0424 

0.0184 

0.017 

0.0444 

0.0251 

0.0324 

0.0445 

gtWiia 

0.0442 

0.0253 

0.0226 

0.0353 

0.0125 

0.054 

0.0644 

0.0336 

0.0357 

0.0185 

0.0358 

0.0426 

0.0238 

0.018 

0.0212 

0.0277 

0.0229 

0.0393 

0.0403 

0.0289 

0.02 

0.0307 

0.0316 

0.0498 

0.046 

0.0143 

0.0162 

0.0148 

0.0132 

0.0162 

0.017 

0.0155 

0.0091 

0.0103 

0.0155 

0.0051 

0.0048 

0.0056 

0.005 

0.0088 

0.0249 

0.0115 

0.0131 

TOillHililiUti 

0.0129 

0.012 

0.0065 

0.006 

0.0073 

0.0099 

0.007 

0.0071 

0.0014 

0.0018 

0.0028 

0.0015 

0.0026 

0.0019 

0.0216 

0.0392 

0.0257 

0.0321 

0.0209 

0.0198 

0.0221 

0.0221 

0.0179 

0.0133 

0.0127 

0.0082 

0.0077 

0.0114 

0.0092 

0.0067 

0.0096 

0.0114 

0.0065 

0.0092 

0.0081 

0.0079 

0.0017 

0.0015 

0.0031 

0.0005 

0.0006 

0.017 

0.025 

0.017 

0.0131 

Design  Point 
Reference  Number 


157 


158 


159 


160 


161 


162 


163 


164 


165 


166 


167 


168 


169 


170 


171 


172 


173 


174 


175 


176 


177 


178 


179 


180 


181 


182 


183 


184 


0.0061 


0.0103 


0.0155 


0.0026 


0.0119 


0.0077 


0.0074 


0.0032 


0.0036 


0.0125 


0.0029 


0.006 


0.0023 


0.0057 


0.0134 


0.0185 


0.0161 


0.0076 


0.0026 


0.0131 


0.0274 


0.0093 


0.0078 


Ethridge 

MRand 

0.0038 

0.0118 

0.007 

0.0088 

0.0087 

0.0066 

0.0028 

0.0029 

0.003 

0.0071 

0.0061 

llBgEI 

0.0035 

0.0021 

0.0035 

0.0044 

0.0034 

0.0071 

0.0233 

0.0111 

0.0032 

0.0047 

0.0024 

0.0027 

0.0029 

0.0039 

0.0023 

Rayleigh\Numerical\  TMCB 


EH3I 


0.0202 


0.0229 


0.0084 


0.0134 


0.0036 


187 

0.0042 

188 

0.0024 

189 

0.0035 

0.0041 


0.0158  0.0078 


mssmssmi 


0.0013 


0.0101 


0.0033 


0.0049 


0.0025 


0.0026 


0.0033 


0.034 


0.0029 


0.0018 


0.0023 


0.0034 


0.0028 


0.0019 


0.0036 


0.0018 


0.0021 


0.0047 


0.0225 


0.0084 


0.0015 


0.0047 


0.0074 


0.0038 


0.0035 


0.0028 


0.0117 


0.0064 


0.0039 


0.0039 


0.0018 


0.0054 


0.0077 


0.0323 


0.0291 


0.0052 


0.0057 


0.0167 


liiiyM 


0.0137 


0.0132 


0.0051 


0.0024 


0.0016 


0.0127 


0.0093 


0.0023 


0.0044 


0.0119 


0.0108 


0.0092 


0.0088 


0.0061 


0.0085 


0.0254 


0.0233 


0.0231 


0.0127 


0.0015 


0.0062 


0.0039 


0.0047 


0.0162 


0.0122 


0.0026 


0.0054 


0.0142 


0.0101 


0.0112 


0.0092 


0.0117 


0.0306 


0.0278 


0.0237 


0.0191 


0.0011 


0.0047 


0.0033 


0.0044 


0.0178 


0.002  0.0042 


0.0062  0.0039 


0.0071 


0.0112 


0.0119  0.0117 


IB 


BS 


0.0164  0.0148 


0.0168  0.0105 


0.0168  0.0131 


0.0034 


0.0066 

0.0069 

0.0059 

0.0116 

0.0062 

0.0053 

0.003 

0.004 

0.0066 

0.0053 

0.012 

0.0169 

0.005 

0.0031 

0.0051 

liiS 


0.0017  0.0051 


0.0031  0.0029 


0.0039  0.0038 


0.0061 


0.0077 


0.0093 


0.0039  0.0023 


0.0086  0.0075 


0.0065  0.0034 


0.0079 


0.005 


0.004 


0.0044 


0.0024 


0.007 


0.0046 


0.0067  0.0059 


0.001  0.001 


0.0043 


BS 
BI 

Blio 


0.0157 


0.0041 


0.002 


0.0019 


0.0016 


0.008 


0.0103 


0.0087 


0.0022 


0.0024 


0.0086 


0.0 


0.0078 


0.0035 


0.0094 


0.0174 


0.0181 


0.0165 


0.001 


0.0018 


0.0025 


0.0029 


0.0032 


0.0052 


0.0049 


0.0042 


0.0076 


0.0024 


0.0137 


0.0048 


0.0019 


0.0019 


11. 

76 


BBBi 

Bi 

LWira 


0.0024 


0.0023 


0.009 


0.0115 


0.013 


0.0075 


0.0034 


0.0104 


0.0166 


0.017 


0.0158 


0.0 


0.0009 


0.0031 


0.0034 


0.0052 


0.0055 


0.002 
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Design  Point 
Reference  Number 


197 


198 


199 


200 


201 


Smed  \Ethridge\MRand\Valstar  Grubbs \Rayleigh 


0.0146 


0.0046 


0.0175 


0.0068 


0.0123 


0.0129 


0.008 


0.0109 


0.0088 


0.0204 


0.0123 


0.0192 


0.0111 


25 


0.012 


0.0047 


0.0026 


0.0054 


0.0015 


0.0003 


0.0114 


0.0041 


0.0032 


0.0029 


0.0004 


0.0086 


0.0024 


0.0023 


0.0022 


0.0006 


0.0019 


0.0024 


0.0009 


0.0004 


0.0004 


0.0031 


0.003 


0.0014 


0.0007 


0.0027 


0.0032 


0.0086 


0.0186 


0.0213 


0.009 


0.0099 


0.0067 


0.0262 


0.0381 


0.0075 


0.0083 


0.0052 


0.0033 


0.0007 


0.0003 


0.0097 


0.0036 


0.0047 


0.0011 


0.0003 


0.0097 


0.0025 


0.0017 


0.0022 


0.0008 


0.002 


0.0006 


0.0005 


0.0003 


0.0021 


0.0023 


0.0027 


0.0018 


0.0009 


0.0023 


0.0018 


0.0062 


0.0054 


0.0109 


0.013 


0.0058 


0.009 


0.016 


0.0273 


0.0161 


0.0068 


0.0116 


0.0202 


0.0345 


0.0106  0.0081 


0.0175  0.0145 


0.034  0.0201 


0.0109  0.0091 


0.0137  0.0089 


0.0226  0.0181 


0.0086 


0.0084 


0.0128 


0.0151 


0.0095 


0.0084 


0.0069 


0.0155 


0.0154 


0.0102 


0.0145 


0.1212 


0.1431 


0.012 


0.0209 


0.0276 


0.076 


0.1198 


0.0651 


0.0344 


0.0594 


0.173 


0.1958 


0.1381 


0.04 


0.0456 


0.1307 


0.081 


0.0167 


0.0099 


0.0289 


0.1395 


0.0711 


0.0052 

0.0069 

0.0024 

0.006 

0.0034 

0.0044 

0.0008  0.0009 


0.0006 


0.002 


0.0081 


0.0043 


0.0028 


0.0037 


0.0017 


0.0021 


0.0022 


0.0017 


0.0026 


0.0004 

0.0029 

0.0027 

0.0012 

0.0021 

0.0036 

0.0022 

0.0051 

iifiWUlifiMi 

0.004 

0.0037 

0.0114 


0.0044 


0.0077 


0.0145 


0.0212 


0.012 


0.0034 


0.0095 


0.0199 


0.0276 


[ililiTillliiiiiUBigtiinii 


0.0022  0.003  0.0069 


0.0033  0.0025  0.0046 


0.003 


igpis 


0.0043  0.0043 


0.001  0.0011 


0.0027  0.0022 


0.0026  0.0026 


0.0023 


36 


0.0006 


0.0002 


0.0025 


0.001 


0.0001 


0.0039 


0.0011 


0.002 


0.0021 


0.0002 


0.0003 


0.0002 


0.0021 


0.0015 


0.0008 


0.0018 


0.0077 


0.015 


0.021 


0.0114 


0.0032 


0.0094 


0.0282 


0.0158 


0.0051 


0.0022 


0.0007 


0.0024 


0.0001 


0.004 


0.0011 


0.002 


0.0019 


0.0001 


0.0005 


0.0002 


0.0006 


0.0024 


0.0024 
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Design  Point 
Reference  Number 


237 


238 


239 


240 


241 


242 


243 


244 


245 


246 


247 


248 


249 


250 


251 


252 


253 


254 


255 


Smed  \Ethridge\MRand\Valstar  Grubbs  \Rciyleigh 


0.0057 


0.0022 


0.0035 


0.0007 


0.0028 


0.0021 


0.006 


0.0027 


0.0018 


0.004 


0.0092 


0.014 


0.0025 


0.0021 


0.0013 


0.0009 


0.0013 


0.0014 


0.0056 


0.0011 


0.006 


0.0066 


0.0027 


0.0012 


0.0034 


0.0053 


0.0062 


0.0012 


0.0032 


0.0045 


0.0032 


0.0032 


0.002 


0.0007 


0.0008 


0.0008 


0.0032 


0.0072 


0.0224 


0.1027 


0.0815 


0.0213 


0.0209 


0.0286 


0.1674 


0.3311 


0.0326 


0.0799 


0.0367 


0.1178 


0.3554 


0.0632 


0.0344 


0.0287 


0.2037 


0.2108 


0.0019 


0.0021 


0.0011 


0.0019 


0.002 


0.0039 


0.0014 


0.0037 


0.0024 


0.0072 


0.0079 


0.006 


0.0018 


0.0022 


0.0044 


0.001 


0.0038 


0.0038  I  0.0045 


0.008 


0.0052 


0.0066 


0.0085 


0.0039 


0.0025 


0.0054 


0.008 


0.0048 


0.0049  0.0044 


0.0016 


0.0031 


piiiwaBiiiii 
riiiiiiiraiiBBSaroiffi 


0.0009  0.001  0.0037 


0.001  0.0013  I  0.0018 


0.0031 


0.0043 


27 


0.0022 


0.0004 


0.0017 


0.0019 


0.0042 


0.0011 


0.0022 


0.0025 


0.004 


0.0075 


0.0039 


0.0031 


0.0008 


0.0007 


0.0009 


0.0046 


0.0052 


0.0025 


0.0037 


0.0033 


0.0113 


0.007 


0.0012 


0.0132 


0.0033 


0.0059 


0.0023 


0.0064 


0.0051 


0.0009 


0.0074 


0.0052 


0.0008 


0.0041 


0.004 


0.008 


0.0042 


0.0591 


0.1054 


0.1056 


0.0986 


0.0219 


0.0336 


0.0525 


0.1981 


0.1021 


0.0317 


0.036 


0.3323 


0.005 


0.0064 


0.0013 


0.0014 


0.0033 


0.0086 


0.0074 


0.001 


0.0043 


0.0037 


0.0046 


0.0053 


0.0013 


0.0074 


0.0063 


0.0018 


0.0033 


0.007 


0.0062 


0.0086 


0.0042 


0.0021 


0.0042 


0.0011 


0.0021 


0.0026 


0.0008 


0.0007 


0.0009 


0.0042 


0.0005 


0.0046 


0.0004 


257 

0.0014 

0.0012 

0.0358 

0.0009 

0.0011 

0.0048 

0.0012 

0.0013 

258 

0.0018 

0.0012 

0.0202 

0.0007 

0.0011 

0.0038 

0.001 

0.0011 

259 

0.004 

0.0026 

■iKiiblSIitmiViU 

0.0024 

0.0039 

0.0021 

mm 

260 

0.0029 

0.0015 

0.0971 

0.0015 

0.0013 

0.0038 

0.0013 

MiMH 

261 

0.0023 

0.0019 

0.1107 

0.0019 

0.0021 

0.0018 

0.0014 

0.0014 

262 

0.0028 

0.0019 

0.0155 

0.0018 

0.0017 

0.0044 

0.0017 

lifililM 

263 

0.0056 

0.0029 

0.0098 

raiMiikixwiitiia 

0.0062 

0.0012 

MiIilEI 

0.0009 


0.0037 


0.002 


0.0056 


55 


0.0008 


0.0011 


0.003 


0.0073 


0.0058 


0.0011 


0.004 


0.0061 


0.0 


0.0008 


0.0013 


0.003 


0.007 


0.0056 


0.0014 


0.0039_ 
26 


0.00^ 
44 
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Sample  Size  9 


Design  Point 
Reference  Number 


Smed  \Ethridge\MRand\ 


0.0968 


0.0245 


0.0258 


0.0198 


0.0149 


0.0832 


0.0762 


0.0443 


0.0328 


0.0728 


0.0633 


0.0814 


0.0457 


0.0624 


0.0224 


0.0711 


0.0342 


0.0269 


0.0327 


0.0311 


0.0491 


0.0632 


0.0425 


0.0257 


0.0351 


0.0887 


0.102 


0.0235 


0.0244 


0.0329 


0.0386 


0.026 


0.0175 


0.0218 


0.0148 


0.0412 


0.0408 


0.0462 


0.0295 


0.0177 


0.0183  0.0135 


0.0661  0.0563 


0.0716  0.0677 


0.043  0.0477 


Rayleigh 

Numerical 

0.0609 

0.0238 

0.017 

0.0174 

0.0209 

0.0226 

0.016 

0.014 

0.0655 

0.0551 

0.0231 


0.018 


0.0166 


0.0105 


0.0467 


0.0353 


0.0188 


0.0228 


0.0514 


0.0157 


0.0371 


0.0275 


0.0449 


0.02 


0.0399 


0.0293 


0.0532 


0.0176 


0.0402 


0.0285 


0.0592 


0.0307 


0.0288 


38 


0.0484 


0.0153 


0.0333 


0.0181 


0.0384 


0.103 


0.0671 


0.0163 


0.0323 


0.0496 


i  j. 

0.0393 

0.0365 

0.0107 

0.0175 

0.0153 

0.0178 

0.0664 

0.0481 

0.0704 

0.0888 

0.0517 

0.0444 

0.0285 

0.0205 

0.0173 

0.0478 

0.0377 

0.0144 

0.0854 

0.0238 

0.0292 

0.0659 

0.0512 

0.0638 

0.1157 

0.0566 

0.0336 

0.0679 

0.0234 

0.0209 

0.066^ 

0.0607 

0.0292 

0.0317 

0.0139 

0.0143 

0.1266 

0.0716 

0.0655 

0.0354 

0.0183 

0.0176 

0.0262 

0.0325 

0.0402 

0.0582 

0.0251 

0.0232 

0.0731 

0.0498 

0.0528 

0.0568 

0.0227 

0.0198 

0.0483 

0.037 

0.011 

0.0599 

0.0471 

0.0427 

0.0306 


0.0128  0.0121 


0.0234 


0.0545 


0.0131 


0.0372 


0.0207 


0.0266 


0.0171 


0.022 


78 


0.0121 


0.0305 


0.0282 


0.0383 


0. 


Bli! 


0.0116 


0.027 


0.0221 


0.0337 


MPX 

5181 

5181 


0.0191 


0.024  0.0153  I  0.0177 


0.0185  0.0139 


0.018 


0.0362 


0.0173 


0.0309  0.0278 


0.0125  0.0124 


0.014  1  0.012 


0.0593 


0.0463 


59 


0. 


liliHiEl 
1818! 
BBl 


0.0139 


0.0229 


0.0898  I  0.0645 


0.0401 


0.078 


0.0266 


0.0312  0.0311 


0.0214  0.0146 


0.0897  0.0589 


0.0257  0.0187 


0.0574 


0.0296 


0.0878 


0.0337 


0.039 


0.0324 


0.0292 


0.0169 


0.0295 


0.0112 


0.0551 


0.0164 


0.0336 


0.017 


0.0637 


0.033 


Sliiw 

BE 


0.0172 


0.0272 


0.0085 


77_ 

25 


Pig 

BI8I 
B185 

WtlEEl 


0.0393 


0.0248  0.0199 


0.0138  0.0141 


0.0403  0.0339 
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Design  Point 
Reference  Number 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


Smed 


0.0366 


0.0516 


0.1341 


0.0326 


0.0765 


0.0469 


0.0258 


0.1272 


0.0731 


0.0367 


0.0351 


0.057 


0.0334 


0.0571 


0.0181 


0.0358 


0.0386 


0.0198 


0.019 


0.0057 


0.0583 


0.0396 


0.0514 


0.0298 


0.0253 


0.0537 


0.0093 


0.0571 


0.0177 


0.021 


0.0117 


0.033 


0.0532 


0.0145  0.0151  0.0348 


0.0829  0.0599  0.0704 


0.0189  0.0165  0.0206 


0.0296  0.027  0.0216 


0.0173 


0.037 


0.0732  0.0612 


0.0968 


0.0331  0.037  0.031 


0.0228  0.0253  |  0.018 


0.07241  0.116  0.0903 


0.0159  0.012  0.0204 


0.0391 


0.0284 


0.0732 


0.0432 


32 


0.0314 


0.0596 


0.0285 


0.0303 


0.0173 


0.008 


0.0124 


0.0561 


0.0167 


289 


0.0152  0.0106 


PIBi 

BBI 

UIIMIX 


0.025 


0.0319 


0.0184 


0.0873  I  0.0763 


0.0478 

0.0342 

0.0347 

0.0499 

0.0373 

0.0482 

0.0354 

0.0533 

0.0407 

0.029 

0.0374 

0.0302 

0.053 

0.0331 

0.0766 

0.1046 

0.0879 

0.0901 

0.0854 

0.0419 

0.0247 

0.0112 

0.0169 

0.0206 

0.0372 

0.0175 

0.0428 

0.0414 

0.0229 

0.0476 

0.0647 

0.0253 

0.0309 

■tllMMCaWKIlCtlEM 

0.0217 

0.1276 

0.0724 

0.1257 

0.1438 

0.1111 

0.103 

0.1182 

0.0293 

0.0132 

0.0168 

0.0358 

0.0132 

0.0682 

0.0612 

0.036 

0.0319 

0.0419 

0.0463 

0.0413 

0.0382 

0.029 

0.0222 

0.0242 

0.0205 

0.025 

0.0208 

0.1155 

0.0482 

0.031 

0.0395 

0.0237 

0.0891 

0.0459 

0.0276 

0.0412 

0.0245 

0.0369 

0.0224 

0.0603 

0.0198 

0.0221 

0.0213 

0.0523 

0.042 

0.0266 

0.0311 

0.0468 

0.0238 

0.0114 

0.0136 

0.0118 

0.013 

0.0122 

0.0887 

0.0627 

0.0333 

0.0428 

0.0292 

0.0402 

0.0291 

0.0967 

0.0311 

0.0489 

0.0499 

0.0388 

0.0351 

0.0378 

0.0403 

0.026 

0.0194 

0.0239 

0.0174 

0.0154 

0.0055 

0.0116 

0.0154 

0.0136 

0.0163 

0.0276 

0.0416 

0.0334 

0.031 

0.026 

0.0308 

0.0446 

0.0373 

0.0303 

0.0971 

0.0795 

0.1367 

0.1703 

0.1172 

0.1438 

0.1106 

0.1096 

0.0453 

0.0532 

0.0775 

0.0563 

0.0813 

0.0553 

0.0212 

0.0208 

0.0177 

0.0261 

0.0165 

0.1055 

0.0847 

0.0973 

0.1127 

0.076 

0.1135 

0.0699 

0.0306 


0.0331 


0.0207 


0.0246 


0.1102 


MPia 

I^SI 


0.0224 


0.019 


0.0304 


0.0132 


0.0287 


0.0386 


QS 


0.0268 


0.0917 


Design  Point 
Reference  Number 


19 


80 


81 


82 


83 


84 


85 


86 


87 


88 


89 


90 


91 


Smed 

Ethridge  . 

0.139 

0.0467 

0.1299 

0.0482 

0.1333 

0.0413 

0.0212 

0.0246 

0.0482 

0.0176 

0.0315 

iiiiiim 

0.0185 

0.0691 

0.0621 

0.0079 

0.018 

0.0528 

0.0487 

0.0652 

0.0487 

0.0649 

0.0295 

0.0302 

0.047 

0.027 

0.0158 

0.035 

0.0303 

0.0132 

0.024 

0.0422 

0.0207 

0.0212 

0.0162 

0.0378 

0.0211 

0.015 

0.0162 

0.043 

0.0071 

0.0255 

0.0233 

0.0208 

0.0391 

0.0122 

0.0146 

0.0075 

0.0064 

0.0311 

0.0221 

0.0491 

0.0195 

0.0316 

0.038 

0.0599 

0.051 

0.0093 

0.0086 

0.0616 

■littllrll 

0.0654 

0.0158 

0.0278 

0.0313 

0.0269 

0.0647 

0.0262 

0.0467 

0.0736 

0.061 

0.0477 

0.0537 


0.0539 


0.0459 


0.0272 


0.0186 


0.0046 


0.0076 


0.0646 


0.0145 


0.0503 


0.019 


0.0222 


0.0579  I  0.0483 


0.0779  0.0472 


0.0364 


0.0238 


0.0037 


0.0098 


0.0856 


0.0241 


0.0601 


0.0217 


0.0229 


0.0213 


0.0182 


0.0046 


0.0175 


0.0484 


0.0109 


0.0438 


0.014 


0.0188 


0.0345 


0.0685 


0.0393 


0.016 


0.0135 


0.0099 


0.021 


0.0525 


0.0073 


0.0352 


0.0104 


0.0212 


0.028  0.0236  |  0.0249 


0.0152 


0.0323  0.0452 


0.0175  0.0184 


0.0104  I  0.0108 


0.0108 


96 


0.0265 


0.0163 


0.0376 


0.0397 


0.0228 


0.0125 


0.0142 


0.0261 


0.0116 


0.0136 


0.0354 


0.0221 


0.0255 


0.0154 


0.0144 


0.032 


0.011 


0.0139 


0.0458 


0.0068 


0.014 


0.033 


0.01 


0.0106 


0.0335 


0.0192 


0.0274 


0.0106 


0.0128 


0.0236 


0.0086 


0.0109 


0.0417 


0.0161 


0.0597 


0.0533 


0.0227 


0.0294 


0.0318 


0.0692 


0.0776 


0.0702 


I®?® 

M_ 

ss^ 


0.0174 


0.0925 


0.0634 


0.0342 


0.0099 


0.0497 


0.0432 


0.0231 


0.0441 


0.0098 


0.0593 


0.0467 


0.0444 


0.0338 


0.0197 


0.0186 


0.0045 


0.012 


0.0108 


0.044 


0.0144 


0.0174 


0.038 


0.0354 


0.0859 


0.0244 


0.0208 


0.0345 


0.0781 


0.0563 


0.0337 


0.017 


0.0071 


0.0081 


0.0069 


0.0406 


0.0308 


0.022 

0.0252 

0.0139 

0.0104 

0.0195 

0.0192 

0.0206 

■WiWiKM 

0.019 

0.0064 

0.0088 

0.0143 

0.0211 

nintil 

0.0058 

0.0067 

0.0138 

0.0151 

0.0305 

0.0229 

0.0233 

0.0237 

0.0072 

0.0085 

0.0424 

0.0325 

0.0442 

0.0418 

0.0158 

0.0143 

0.0239 

0.0213 

0.0228 

0.0219 

0.0593 

0.0512 

0.0519 

0.0465 
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Design  Point 
Reference  Number 


119 


120 


121 


122 


123 


124 


125 


126 


127 


128 


129 


130 


Smed  \Ethridge\MRand\Valstar  Grubbs  \Rayleigh\Numerical 


0.0666 


0.074 


0.0343 


0.0144 


0.0134 


0.0183 


0.0931 


0.0602 


0.0323 


0.0257 


0.0384 


0.0372 


0.0387 


0.0356 


0.0537 


0.0262 


0.0116 


0.0588 


0.0376 


0.2623 


0.0311 


0.0644 


0.041 


0.0893 


0.1168 


0.0358 


0.0111 


0.0315 


0.0227 


0.0019 


0.0473 


0.0545 


0.0532 


0.013 


0.0884  0.0543 


0.0646  0.0519 


0.0354  0.075 


0.0466  0.0474 


0.0167  0.0237 


0.0167 


0.028 


0.037 


0.0259 


0.0344 


0.0229 


0.0185 


0.028 


0.0466 


0.0267 


0.0378 


0.0411 


0.0174 


0.0342 


0.0274 


0.0193 


0.0118 


0.0347 


0.0487 


0.0146 


0.0318 


0.018 


0.0405 


0.0394 


0.0182 


0.0137 


0.0451 


0.0558 


0.0218 


0.0112 


0.0275 


0.0413 


0.0185 


0.0133 


0.0232 


373 


0.0169 


0.0146 


0.0576 


0.028 


0.0327 


0.0453 


0.0733 


0.0606 


0.0393 


0.0189 


0.0218 


0.0109 


0.0028 


0.0502 


0.0346 


0.0291 


0.0093 


0.0058 


0.0242 


0.012 


0.0117 


0.0247 


0.0071 


0.0259 


0.0209 


0.0217 


0.1452 


0.0294 


0.0307 


0.0989 


0.1035 


0.1112 


0.0267 


0.0138 


0.0107 


0.0047 


0.0054 


0.0246 


0.0102 


0.0172 


0.0075 


0.027 


0.0314  I  0.0226 


0.0104 


0.0275 


0.0313  0.0162 


0.0359  0.0132 


0.0442  0.0316 


rixiMRmffyei 


0.1102 


0.1239 


0.0359 


0.0192 


0.0123 


0.0052 


0.0032 


0.0379 


0.0124 


0.0178 


0.0783 


0.0865 


0.0186 


0.0119 


0.0099 


0.007 


0.0031 


0.0318 


0.0117 


0.0212 


0.0197 


0.0104 


0.0261  0.0159 


0.0127  0.007 


0.0462 

0.0512 

0.0414 

0.0534 

0.0476 

0.0396 

0.0179 

0.0135 

0.0166 

0.0168 

0.0108 

0.0067 

0.0045 

0.0121 

0.0106 

Mtlltil 

0.0382 

0.027 

hWrlfM 

0.0481 

0.0313 

■wtwaa 

0.0199 

0.0179 

Millda 

0.0169 

0.0138 

0.0147 

0.0225 

0.036 

0.0235 

0.0333 

0.0353 

mm 

0.0347 

0.011 

0.018 

0.0159 

0.0229 

0.0272 

mm 

0.0375 

0.0206 

mm 

0.0074 

0.0066 

0.0251 

0.0268 

0.0147 

0.0162 

0.0136 

0.011 

0.0125 

0.0102 

0.1061 

0.0974 

0.0813 

0.0415 

0.0259 

0.0225 

0.0291 

0.0234 

0.0228 

0.066 

0.0683 

0.0587 

0.0684 

0.0781 

0.0761 

0.0703 

0.092 

0.0832 

0.0146 

0.0207 

mm 

0.0083 

0.0132 

0.0147 

0.0095 

0.0103 

0.0092 

0.0067 

0.0069 

0.0042 

0.0013 

0.0031 

0.0296 

0.0327 

0.0127 

0.0125 

0.0207 

0.0214 

0.0214 

0.0097 

0.0046 

0.0073 

0.0041 

0.0024 

0.002 

0.0131 

0.0182 

0.0184 

0.0069 

0.0075 

0.0076 

0.0124 

0.0138 

0.014 
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Design  Point 
Reference  Number 


159 


160 


161 


162 


163 


164 


165 


166 


167 


168 


169 


170 


171 


172 


173 


174 


175 


Smed 


0.009 


0.0019 


0.0107 


0.0161 


0.0032 


0.0054 


0.0012 


0.0283 


0.024 


0.0148 


0.0215 


0.0063 


0.0427 


0.0228 


0.0188 


0.0209 


0.0124 


0.0857 


0.0229 


0.0113 


0.0459 


Ethridge 

MRand 

0.011 

0.0158 

0.003 

0.0034 

0.0064 

0.0155 

0.011 

0.0099 

0.0041 

0.0059 

0.0034 

0.0046 

0.0064 

0.0052 

0.0078 

0.0126 

0.0143 

0.0121 

0.0103 

0.0078 

0.0055 

0.0083 

0.0058 

0.0073 

0.0182 

0.0108 

IBiItHEB 

0.025 

IBiftlRfeB 

0.0167 

0.0128 

0.012 

0.0203 

0.0198 

0.0124 

0.0282 

0.0283 

0.0276 

0.0199 

0.0286 

0.0276 

0.0277 

0.0083 

0.0094 

0.0094 

mm\ 

Rayleigh  \Numerical\  TMCB. 


0.0562 


0.0343 


0.0051 


0.0089 


0.0123 


0.0196 


0.0357 


0.0058 


0.0164 


0.0102 


0.0297 


0.0122 


0.0059 


0.0089 


0.0256 


PTII 

EIRI^ 


0.0076 


0.0119 


0.0089 


0.0127 


0.008 


[MPIlg* 

SB! 


68 


0.0334 


0.0015 


258 


0.0097 


0.0059 


0.0027 


0.0046 


0.019 


0.0133 


0.0062 


0.007 


0.0069  0.0041 


0.0035  0.0044 


0.0063  0.0055 


0.0096  0.0093 


0.005  0.01 


0.0059  0.0083 


0.0314 


0.0274 


0.021 


0.0272 


0.0388 


0.0268 


335 


0.029 


0.0098 


0.03 


0.0228  I  0.0155 


0.0202 


0.0139 


Plw 

BBI 


0.0244  0.0182 


0.0216  0.0147 


0.02  0.022 


0.0195 


0.0508 


0.0209 


0.0171 


0.0069 


0.0072 


0.0038 


0.0499 


0.0274  0.041 


0.0145  0.0271 


0.0126  0.0058 


53 


0.0459  0.0384 


0.0265  0.0238 


0.0224  0.0136 


0.0071 


0.0079 


0.0166 


0.0285 


0.0058 


0.0085  0.0074 


0.0053 


0.0149  0.0124 


SfilE! 

BCIgfelWXIMK 


0.0311 


0.0151 


0.0166 


0.0114 


0.0226 


0.0191 


0.0374 


0.0072 


0.0075 


0.0105 


0.0398 


0.0293 


0.0157 


0.0103 


0.0085  0.0056 


0.0109  0.0099 


0.0088  0.0075 


0.0265  0.0167 


0.013  0.0087 


SJRIE&1 


0.0203 


0.0099 


0.0062 


0.0054 


0.003 


0.003 


0.0083 


0.0044 


0.0036 


0.004 


0.0086 


0.0046 

0.0072 

0.006 

0.0073 

0.0039 

0.0064 

0.0251 

0.021 

0.0245 

0.0261 

0.0148 

0.0144 

0.0109 

0.0077 

0.0191 

0.0163 

0.0224 

0.019 

0.0205 

0.0202 

0.0245 

0.022 

0.0283 

0.028 

0.0063 

0.0067 

0.0135 

0.0112 

Iwn^MKannsHii 

0.022 

0.017 

0.0181 

0.0149 

0.0023 

0.0071 

iilifiV/J 

0.0069 

0.0058 

0.0127 

0.0109 

0.0269 

0.0244 

0.0043 

0.0049 

0.011 

0.0084 

0.0089 

0.0062 

0.0281 

0.0215 

0.0089 

0.0046 

0.0076 

0.0064 

0.0091 

0.0106 
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Design  Point 
Reference  Number 


Smed  \Ethridge\MRand\Valstar  GrubhsvRayleigh 


0.0341 


0.0316 


0.005 


0.0261 


0.0503 


0.0738 


0.0762 


0.0173 


0.012 


0.0872 


0.0658 


0.0577 


0.0118 


0.007 


0.0029 


0.0013 


0.0094 


0.0072 


0.0076 


0.0053 


0.0004 


0.0072 


0.0057 


0.0085 


0.0012 


0.0013 


228 


0.0068 


0.0015 


0.0012 


0.002 


0.0087 


0.006 


0.0011 


0.0071 


0.0047 


0.0057 


0.0264 


0.0454 


0.0052 


0.0165 


0.0601 


0.0387 


0.044 


0.1482 


0.0295 


0.0429 


0.0279 


0.0776 


0.2181 


0.046 


0.0287 


0.1058 


0.0621  0.0574 


0.0109 


0.0194 


0.0703 


0.0501 


0.051 


0.0052 


0.0057 


0.0044 


0.0086 


0.0142 


0.0484 


0.0323 


0.0357 


0.0071 


0.0055 


0.0033 


0.0028 


0.051 


0.0673 


0.0338 


0.0072 


0.0179 


0.0489 


0.0473 


0.0306 


0.0159 


0.007 


0.0031 


0.0018 


0.0008 


0.0153 


0.0059 


0.0082 


0.0033 


0.OOO7 


0.0121 


0.0028 


0.0053 


0.0029 


0.0008 


0.0036 


0.0025 


0.0012 


0.0008 


0.0025 


0.002 


0.0029 


0.0031 


0.0004 


0.0042 


0.0013 


0.008 


0.0202  0.0266 


0.005 


0.0163 


0.0335 


0.0196 


0.0033 


0.0467 


0.0383 


0.0048 


0.0134 


0.0411 


0.0249 


0.0301 


0.0076 


0.0089 


0.0033 


337 


0.0577 


0.0483 


0.0027 


0.0129 


0.0515 


0.036 


0.0402 


0.0068 


0.1613  0.0046 


0.0033 


0.001 


0.0055 


RES 


0.2021 


0.0925 


0.1337 


0.0847 


0.0602 


0.3066 


0.1415 


0.0292 


0.0349 


0.0706 


0.3067 


0.0991 


0.0273 


5^ 


0.0004 


0.0055 


0.0027 


0.0012 


0.0058 


0.0033 


0.0011 


0.0043 


0.003 


0.0016 


0.006 


0.0025 


0.0092 


0.0033 


0.0071 


0.0048 


0.0019 


0.0039 


0.0037 


0.0031 


0.0033 


0.0036 


0.0024 


0.0006 


0.0041 


0.0023 


0.0073 


0.0026 


0.0031  0.0034 


0.0024  0.0068 


0.0006  0.003 


0.0045  0.0036 


0.0031 


0.0005 


0.005 


0.0039 


0.0053 


0.0032 


0.0002 


0.0045 


0.0024 


0.0013 


0.0008 


0.0019 


0.0025 


0.0027 


0.0072  0.0096 


0.0069 


H 


32 


0.0012 


0.0073 


0.0041 


0.0289 


0.0199 


0.003 


0.0654 


0.0489 


0.0027 


0.0126 


0.0514 


0.0404 


0.0443 


0.0061 


0.0015 


0.0091 


0.005 


0.0075 


0.0034 


0.001 


0.007 


0.0043 


0.0044 


0.0022 


0.0006 


0.0015 


0.0007 


0.0026 


0.0039 


0.0043 


0.0003 

0.0029 


0.0014 


0.0083 


0.0051 
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Design  Point 
Reference  Number 


239 


240 


241 


242 


243 


244 


245 


6 


7 


248 


249 


250 


251 


Smed 

Ethridge  . 

0.0075 

0.0031 

0.0083 

0.0039 

0.0094 

0.0087 

0.0096 

0.0086 

0.0103 

0.0065 

0.0091 

0.0107 

0.003 

0.0041 

0.0068 

0.0053 

0.0166 

0.0155 

0.0131 

0.0173 

0.0088 

0.0097 

0.006 

0.0045 

0.0007 

0.001 

0.0029 

0.0012 

0.0013 

0.0004 

0.002 

0.0025 

0.0156 

0.008 

litMiVII 

0.0013 

■aiiMwi 

0.004 

0.0025 

0.0017 

0.0097 

0.0047 

0.0073 

0.0061 

0.0036 

0.0043 

0.0054 

0.0038 

0.0103 

0.006 

0.006 

0.0084 

0.0062 

0.0071 

0.0017 

0.0029 

0.0104 

0.0074 

0.0079 

0.0089 

0.0205 

0.021 

0.0068 

0.0113 

0.0028 

0.0019 

0.007 

0.0105 

0.0153 

0.0174 

0.0169 

lEE^I 

0.0085 

liMIMI 

0.0794 


0.0983 


0.0806 


0.0632 


0.0092 


0.0046 


0.0067  0.0058 


0.004  0.0036 


0.0082  0.007 


0.0053  0.0079 


0.2842 


0.0569 


0.0665 


0.2077 


0.8667 


0.2958 


0.0 


0.0048  0.0044 


0.0028  0.0028 


0.0119  0.0114 


SKS! 


0.0102 


0.0048 


0.003  0.0038 


0.0115  0.014 


HiliEEMKilllM 


0.0032  0.0033 


0.0055  0.0052 


0.1253 


0.4829 


0.1192 


0.087 


0.0206 


0.0407 


0.2049 


0.0359 


0.0129 


0.0209 


0.1631 


0.3172 


0.0634 


0.0173 


0.0466 


0.1255 


0.0442 


0.1809 


0.1216 


0.156 


1.1043 


Valstar  Grubhs  \Rayleigh\Numerical 


0.0063 


0.0044 


0.0077 


0.0116 


0.0078 


0.0127 


0.0056 


0.0061 


0.0146 


0.0154 


0.0096 


0.0035 


0.003 


0.0047 


0.0029 


0.0038 


0.0094 


0.0027 


0.0086 


0.0054 


0.0077 


0.0067 


0.0021 


0.0071 


0.0078 


0.0077 


0.0103 


nniiyEEIEI 


0.0057  0.0052 


0.0011  0.0014 


0.0009 


0^027  I  0.0028 
.0 


0.0031  0.003 


0.0011  0.0015 


0.006  0.0049 


0. 


0.0007 


0.0027 


0.008 


0.0009 


0.003 


0.0014 


0.0054 


0.0072 


0.0029 


0.0033 


0.0051 


0.0062 


0.0078  0.007  0.0 


0.0101  0.0088  0.0103 


0.0201 


0.0094  0.0093  0.0108 


0.0031  0.0027  0.0017 


0.0125 


0.0189  0.0186  0.019 


0.0014 


0.0035 


0.0024 


0.0078 


0.0083 


0.0026 


0.0024 


0.0047 


0.0073 


[iTiHWaKKilB _ 


4^ 

73 


0.0089 


0.0068 


0.0095 


0.0142 


0.0098 


0.0017  0.0022 


0.0 


0.0105  0.0098 
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Sample  Size  6 


Design  Point 
Reference  Number 


1 


2 


3 


Smed 


0.1605 


0.0622 


0.1047 


0.1153 


0.258 


Rayleigh  \Numerical\  TMCB. 


0.0603 


0.0542 


0.0305 


0.0517 


0.2008 


0.1028 


0.0355 


0.0879 


0.1397 


0.0882 


0.0731 


imraafliTi 


0.1484  0.041 


0.075  0.0664  0.0664  0.0678  0.0675 


0.0909  0.0851  0.0876  0.0894  0.0878 


0.0475  0.0485  0.0674 


0.2149  0.1524  0.1966| 


0.0824  0.0578  0.0605  0.0587 


0.0281  0.0165  0.0255  0.0139| 


395 


0.14841  0.08571  0. 12041  0.07471 


658 


0.07091  0.04611  0.05441  0.0555| 


0.0601  0.0166  0.0197  0.0141 


0.117 

0.03641  0.05061  0.0837|  0.0554' 


0.0758  0.0302 


0.0711 


0.1005 


0.2112 


0.0655 


0.0995 


0.0547 


0.07 


0.1412 


0.0491 


0.0542 


0.0465  0.0449  0.0458 


0.1104  0.1871  0.1626 


0.0267  0.0389  0.0324 


0.0577  0.0588 


0.075 


0.04951  0.0592]  0.0583 


0.08|  0.10721  0.0917] 


0.0292 


0.0593  0.0654  0.0506 


0.0276  0.0453 


0.0361  0.03761  0.0658]  0.0534] 


0.0864  0.0556 


0.0676 


0.074 


0.02571  0.0533]  0.0663]  0.0435] 


0.05  0.065 


0.1142  0.0963 


0.0874  0.0758 


0.0939  0.0572  0.0636  0.0592 


0.0655 


0.0088 


0.1361 


0.0967 


0.0722] 


0.0313 


0.1234 


0.077 


0.0679 


0.0236 


0.0409 


0.057 


0.2124 


0.0412] 


0.0702 


0.0705] 


0.0568 


0.0698 


0.148 


0.049 


0.0417 


0.0505] 


0.0473 


0.0687 


0.0937 


0.0767 


0.0359 


0.0855 


0.1337 


0.0658 


0.068 


0.0888 


0.0495 


0.1474 


0.1134 


0.0584 


0.0139] 


0.0719 


0.0657 


0.0542 


0.0142 


0. 


0.0533 


0.0378 


0.0273 


0.0493] 


0.0457 


0.1592 


0.0324 


0.0739 


0.0366 


0.0463 


0.0604 


0.0811 


0.0487 


0.1225 


0.0642 


0.0151 


0.0506 


0.0607 


0.0327 


0.0228 


0.0572 
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Design  Point 
Reference  Number 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


Smed  \Ethridge 


0.0851 


0.0784 


0.1075 


0.0378 


0.0332 


0.0358 


0.0516 


0.1347 


0.065 


0.0426 


0.0757 


0.1171 


0.0188 


0.1186 


0.2085 


0.0631 


0.1466 


0.071 


0.0294 


0.0524 


0.0292 


0.024 


0.0403 


0.063 


0.0996 


0.0696 


0.0715 


0.0637 


0.0988 


0.2814 


0.0569 


0.0913 


0.1981 


0.1535 


0.1192 


0.0526 


0.0426 


r«ig> 

BIS 


0.0437 


0.0487 


0.0728 


0.0827 


0.0602 


0.0471 


0.0539 


0.0357 


0.0845 


0.1293 


0.0615 


0.0866 


0.0397 


0.0397 


0.0533 


0.0855 


0.0354 


0.0303 


0.0317 


0.0248 


0.0771 


0.0694 


0.0663 


0.0525 


0.0685 


0.0528 


0.1557 


0.0631 


0.0243 


0.0396 


0.0237 


0.0662 


0.0786  0.1083 


0.01911  0.0258 


0.0323 


0.038|  0.0374  0.0401 


0.053  0.06581  0.0537 


0.0445 


0.0652 


0.0995 


0.0631 


0.1533 


0.0734 


0.07331 


0.046 


0.0364 


0.056 


0.0742 


0.0405 


0.037 


0.0215 


0.0341 


0.034 


0.0679 


0.0272 


0.0478 


0.0687 


0.0911 


0.2171 


DjiK_ 

BkiBas 


0.184 


0.0785 


0.1102 


0.0566 


0.0387 


a 

niSt 
SIBI 


0.1411 


0.057 


0.0848 


0.0463 


0.0373 


0.0747  0.0656 


0.0474 


0.0236 


0.0448 


Rayleigh 

Numerical 

0.0368 

0.0223 

0.0558 

0.0614 

0.1217 

0.0878 

0.024 

0.0255 

0.0283 

0.0346 

0.0358 

0.0389 

0.0553 

0.0465 

0.128 

0.1196 

0.0469 

0.0404 

0.0536 

0.0673 

0.0437 

0.0516 

0.0746 

0.0833 

0.0489 

0.0566 

0.0994 

0.0746 

0.1615 

0.1349 

0.0705 

0.0535 

0.111 

0.0842 

0.0577 

0.0348 

0.0577 

0.0454 

l■EBE 

0.0649 

0.0423 

1  0.0436 

0.031 

0.0235 


0.028 


0.0915  0.0611 


0.0548 


0.0684 


0.0924 


0.0507 


0.0843 


0.0741 


7921 


0.0816 


0.0718 


0.0512  0.0597 


0.1663  0.1998 


0.068 


0.058  0.0711 


0.0478 


0. 


0.02441  0.01951  0.0137 


0.0257  0.0284 


0.0492  0.0336 


0.0825  0.05741  0.0516 


0.0698 


0.0714 


0.1794 


0.0942 


0.0155 


0.021 


.0728 


0.2121  0.1653 


0.0568  0.0417 


0. 


0.04791  0.0337 


0.1307  0.1072 


0.1383 


0.0841 


78 

0.1486 

0.0784 

0.1517 

0.1734 

0.1268 

0.1003 

0.1295 
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Design  Point 
Reference  Number 

Smed 

Ethridge 

MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

i^nli 

79 

0.1467 

0.088 

0.1177 

0.1204 

0.1069 

0.1073 

0,1044 

80 

0.0773 

0.0925 

0.0904 

0.1305 

0.116 

0.1144 

0.1183 

■ius^ 

81 

0.0442 

0,0192 

0.0458 

0.0686 

0.0373 

0.0238 

0.0364 

0.034 

82 

0.068 

0.0452 

0.06 

0.0788 

0.0429 

83 

0.0249 

0.0169 

0.0171 

0.0187 

0.0191 

0.0199 

0.0221 

84 

0.0525 

0.05 

0.0307 

0.0334 

0.0394 

0.0339 

0.0405 

0.0386 

85 

0.0176 

0.017 

0.0131 

0.0232 

86 

0.0565 

0.0676 

0.0595 

■wy/aa 

0.0458 

0.037 

0.0443 

0.0411 

87 

0.0339 

0.0367 

omei 

0.0213 

0.0167 

0.0198 

0.0137 

88 

0.0385 

0.0187 

0.0312 

0.0386 

0.0299 

0.0204 

0.0315 

0.0252 

89 

0.04 

0.0208 

■w>iriiriw»Vi:V/i 

0.0208 

Qmii 

0.0221 

0.0205 

90 

0.0271 

0.0232 

0.0273 

0.0337 

0.026 

0.03 

0.0221 

0.0209 

91 

0.0985 

0.0841 

0.0668 

0.0791 

0.0598 

0.0685 

0.0398 


0.0452 


0.027 


0.0255 


0.0289 


0.0362 


0.025 


0.0168 


0.0175 


0.0289 


0.0289 


0.0273 


0.0337 


0.0467 


0.0323 


0.038 


0.0624 


0.0271 


0.0695 


0.0892 


0.04161 


0.0851 


0.1127 


0.0225 


0.0091 


0.0554 


0.0342 


0.0252 


0.0131 


0.0175 


0.0538 


0.0278 


0.0168 


0.0399 


0.0514 


0.0653 


0.0307 


0.0574 


0.0251 


0.0486 


0.1192 


0.0117 


0.0498 


0.0742 


0.0409 


0.049 


0.0452  0.0303 


0.0217 


0.0418 


0.0809 


365 


0.027 


0.0289  0.023 


0.0244  0.0196 


0.0151  0.0154 


0.0258  0.021 


0.0266 


0.0069 


0.0225 


0.0601 


0.0359 


0.0178 


0.0174 


0.0284 


0.0098 


0.0201 


0.0387 


0.027 


0.0132 


0.053 


0.024 


0.0229  0.0235 


twwariMrfgBbK 


0. 


0.0171  0.012 


0.0397  0.0353 


0.0857  0.0549 


0.0528 


0.0531 


BBEHtok 


0.0517  0.0425 
0.0274  0.025 


0.0504  0.0508 


0.05721  0.0456 


0.1734 


0.028 


0.0704 


[£ 

Bks 


0.0587  0.0448 


0.0878  0.0813 


0.0507  0.038 


0.0885  0.0347 


0.0411  0.0435 


0.1162  0.1058 


0.0 


0.0458 


0.0733 


0.0337 


0.0367 


IKIlWill 


725 


0.03021  0.0245 


284 


0.0709  0.0612 


149 


Design  Point 
Reference  Number 


119 


120 


121 


122 


123 


124 


125 


126 


127 


128 


Smed  \Ethridge\MRand 


Rayleigh  \Numerical 


0.0975 


0.0888 


0.0313 


0.0337 


0.0567 


0.0569 


0.1158 


0.0124 


0.0353 


0.0271 


0.0334 


0.058 


0.034 


0.0715 


0.0655 


0.0477 


0.0654 


0.0355 


0.0475 


0.0382 


0.0116 


0.01 


0.0069 


0.0797 


0.0353 


0.0156 


0.0171 


0.0066 


0.0285 


0.0414 


0.0565 


0.0777 


0.0425 


0.0257 


0.16381  0.1793  0.1268 


0.0659  0.07541  0.0516 


0.0954 


0.118 


0.0228 


0.0399 


0.1499 


0.0582 


0.0736 


0.0189 


0.032 


0.0553 


0.0257 


0.0498 


0.0552 


0.0199 


0.0396 


0.04281  0.0509 


939 


0.0095  0.0144 


0.0169  0.0166 


0.0194  0.01541  0.0136 


0.0404 


0.026 


0.0641  0.0384 


0.0334  0.037 


0.0426  0.0552 


0.1325 


0.0676 


0.0524 


0.0359 


0.0606 


0.1136 


0.1191 


0.0589 


0.0742 


0.0135 


0.0292 


0.0068 


0.0039 


0.0392 


0.0303 


0.0196 


268 


0.0558 


0.0243  0.0251 


0.019 


0.02371  0.02761  0.0247 


0.057  0.0474 


0.0568  0.057 


0.08661  0.10421  0.066 


0.0418 


0.0514 


0.1843 


0.1613 


0.0513 


0.0265 


0.0209 


0.0135 


0.003 


0.0065 


0.0474 


0.0271 


0.0203 


0.0126 


0.1901  0.1369 


0.0636  0.0423 


0.0352  0.0222 


0.028  0.0147 


0.0035  0.004 


0.0036  0.0044 


0.065  0.0364 


imiwiaBil&EE 


0.011  0.008 


0.048  0.0384 


0.1124 

0.0507 

IHSEEB 

0.0327 

0.0234 

0.0166 

0.0151 

0.0225 

0.031 

0.0556 

0.0506 

0.0739 

0.09 

0.0226 

0.0124 

0.021 

0.0217 

0.0214 

0.013 

0.0274 

0.0261 

0.0508 

0.0311 

0.0238 

■arm 

0.0398 

0.0438 

0.0127 

0.0204 

0.0399 

0.0428 

0.0249 

0.017 

0.0276 

0.024 

0.0478 

0.0463 

0.068 

0.0644 

0.0786 

0.101 

Mtglrfd 

0.0343 

0.0463 

0.1386 

0.1549 

0.1324 

0.1352 

0.0493 

0.0442 

0.0207 

0.0232 

0.0111 

0.016 

0.0189 

0.0156 

0.0062 

0.0045 

0.03 

0.0384 

0.0187 

0.0218 

0.0161 

0.0025 

0.0197 

0.0361 

0.0281 

0.0409 

0.0446 

0.0433 

0.011 


0.017 


0.0377 


0.0485 


0.0583 


0.0928 


0.1376 


0. 


0.0223 


0.0115 


0.0172 


0.019 


0.006 


0.041 
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Design  Point 
Reference  Number 


159 


160 


161 


162 


163 


164 


165 


166 


167 


168 


169 


170 


171 


172 


173 


174 


175 


176 


177 


178 


179 


180 


Smed  \Ethridge\MRand\Valstar  Gruhhs\Rayleigh 


0.0234 


0.0063 


0.0207 


0.0085 


0.0195 


0.0132 


0.0381 


0.0356 


0.0256 


0.0202 


0.0104 


0.0273 


0.0482 


0.0272 


0.0121 


0.0377 


0.0088 


0.0381 


0.0252 


0.03 


0.0668 


0.0323 


0.1291 


0.0683 


0.0156 


0.0191 


0.0059 


0.0069 


0.0088 


0.0138 


0.0113 


0.0083 


0.0296 


0.0198 


0.0224 


0.0117 


0.0107 


0.0195 


0.0171 


0.0318 


0.015 


0.0077 


0.0141 


0.0256 


0.0505 


0.0905 


0.0382 


0.0289 


0.0279 


0.0439 


0.0844 


0.0364 


0.0228 


0.0052 


0.0062 


0.0105 


0.012 


0.0271 


0.006 


0.0143 


0.0183 


0.022 


0.0528 


0.0184 


0.0495 


0.025 


0.0258 


0.0043 


0.0317 


0.0101 


0.0115 


0.0076 


0.0204 


0.028 


0.029 


0.0134 


0.0081 


0.0333 


0.013 


0.0355 


0.0168 


0.0236 


0.0196 


0.0409 


0.0656 


0.076 


0.0255 


0.0224 


0.0743 


0.0453 


0.0791 


0.0263 


0.0554 


0.0069 


0.0055 


0.0172 


0.0023  0.0062 


tiiik 

REE 


flIO 

SB! 


0.0055  0.0081 


0.0341  0.0307 


0.0348  0.0231 


0.0224  0.0187 


0.0189  0.01671 


0.0168  0.0091 


0.0469  0.0328 


0.0209  0.0145 


0.0242  0.0195 


nittiepi»Tttegj 


0.0493  0.0447 


0.0637  0.0412 


0.0588  0.035 


0.1266  0.1052 


0.0739 


0.0065 


0.0088 


0.0063 


0.0327 


0.0251 


0.023 


0.0354 


0.0154 


0.0095  0.0106 


0.0745  0.0596 


QP 

Q 

RE 


0.034 


BP 

RE 


0.02 


0.0038 


0.0225 


0.0063 


0.0113 


0.009 


0.0252 


0.0135 


0.0103 


0.0186 


0.0139 


0.027 


0.0128 


0.0086 


0.0146 


0.0289 


0.0449 


0.0699 


0.03 


0.0271 


0.0366 


0.0275 


0.0836 


0.0401 


iiiitfaMiititBiaMliTiTg 


0.0126  0.0113  0.0122 


0.0138  0.0154 


0.04191  0.0321  0.0278 


0.007 


lililiHMiIim 


0.0 


0.0132 


0.0229 


0.0459 


0.0061 

0.0084 

0.011 

0.0126 

0.0065 

0.0087 

0.023 

0.016 

0.0213 

0.0196 

0.00^ 

0.0086 

^eeeseebbi 

0.0345 

0.0287 

■wiiiiad 

EililEil 

0.0626 

0.0539 

0.0333 

E>g*!gll 

0.0388 

0.0409 

■IlIKKiM 

0.0374 

0.0478 

0.038 

0.0247 

0.0215 

0.0032 

0.0046 

0.0047 

0.0076 

0.0099 

0.0108 

0.0122 

0.0124 

RE 


0.0095  0.0137 


SElKinsi 


0.0262 


Design  Point 
Reference  Number 


199 


200 


201 


202 


203 


204 


205 


206 


207 


208 


209 


210 


211 


212 


213 


214 


215 


216 


217 


218 


Smed 

Ethridge 

0.0373 

0.0455 

0.0809 

0.0397 

0.009 

0.0163 

0.0541 

0.0439 

0.0573 

0.0293 

0.1002 

0.1081 

0.1075 

0.064 

0.012 

0.0208 

0.0486 

0.0297 

0.0787 

0.0606 

0.118 

0.0692 

IEB 

0.0767 

0.0159 

0.0192 

0.0032 

0.0075 

mm 

hkeh 

0.0225 

0.0038 

■muiggi 

0.0116 

0.0104 

0.0048 

0.0015 

■titniw 

0.0194 

0.0077 

0.0065 

0.0096 

0.0129 

0.004 

0.0043 

0.0015 

0.002 

0.0052 

0.0039 

0.0032 

0.003 

0.0017 

0.0032 

0.0057 

0.0076 

0.0069 

0.0051 

0.0069 

0.0054 

0.0015 

0.0107 

0.0109 

0.0093 

0.0074 

0.008 

0.0057 

0.0306 


0.0559 


0.0155 


0.0331 


0.0738 


0.0745 


0.0338 


0.0751 


1.3283 


0.0352 


0.0762 


0.0193 


0.0296 


0.0503 


0.0188 


0.0952  0.0741 


0.0663 


0.0584 


0.0487 


3.2502 


0.1148 


0.1255 


0.29081 


1.6273 


0.0482 


652 


0.0811 


SK 


0.0596 


0.013 


imiwis 

BBS! 


0.0134 


0.0058 


0.00041  0.00091 


Sfcim 


0.0037 


6.2056 


0.974 


pBBI 
Bl&t 


1 


0.3031 


0.2616 


0.0334 


0.0137 


0.0369 


0.0317 


0.0235 


0.0342 


0.0497 


lili] 

Btog] 


0.0081  0.0066 


0.0137  0.0129 


0.0042  0.0044 


0.00281  0.0035 


0.0051 


0.0078  0.0071 


0.00581 


0.0071  0.0078 


0.0145 


0.0048 


0.0073 


0.0091 


0.0027 


0.0181 


0.0063 


0.0128 


0.0071 


0.0029 


0.0165 


0.0078 


0.0134 


0.0039 


0.0013 


0.0033 


0.0039 


0.0047 


0.0044 


0.0027 


0.0073 


0.0066 


0.0052 


0.0046 


0.005 


0.0104 


0.0091 


0.0087 


0.0294 


0.0492 


0.0334  0.0299 


0.0 
0,0 

0.0129|  0.0178 
0.0333  0.0389 


0.0152  0.0192 


0.06341  0.0816 


0.0602 


0.097 


BP 

E 

Bp 


liMii 

bPB! 


0.0071 


0.0049 


0.0038|  0.0037 
0.0004  0.0026 


0.0057  0.0071 


BSP 

QB^ 
BB 


0.0073  0.013 


0.0142  0.0182 


0.0011  0.0039 


0.0064  0.0111 


0.0029 


0.0013 


SPHTte 


0.0089  0.0112 


0.0058  0.0086 


0.0055  0.0092 


0.0091  0.0075 


078 


152 


Design  Point 
Reference  Number 


239 


240 
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0.0046 


0.0054 


0.0066 


0.0079 


0.0177 


0.0156 


0.0095 


0.0126 


0.0065 


0.0541 


0.0161 


0.0076 


0.0013 


0.0023 


0.0055 


0.0067 


0.0128 


0.0016 


0.0053 


0.0103 


0.0124 


0.0329 


0.0035 


0.0115 


0.0147 


0.0159 


0.0207 


0.0039 


0.0115 


0.0148 


0.0214 


0.0306 


0.0077 


0.0074 


0.0307 


0.0427 


0.0303 


0.0041  0.20961  0.0025  0.0022 


0.0058  2.1581  0.00721  0.0063 


0.3653 


0.0442  0.00631  0.0045 


Rayleigh 


0.0071 


0.0073 


0.0165 


0.017 


0.0067 


0.0095 


0.0026 


0.0396 


0.0175 


0.0098 


0.0023 


0.0023 


0.0054 


0.0028 


0.0083 


0.0017 


0.0043 


0.0091 


0.0054 


0.0198 


0.0021 


0.0131 


0.0169 


0.013 


0.0201 


0.0033 


0.0099 


0.0132 


0.0227 


0.0239 


0.0037 


0.0076 


0.0362 


282 

185 


0.3862 


0.26881  0.0164  0.0159 


0.00791 


O.Oll  0.0 


0.25061  0.0063 


1.3309  0.0434  0.0407| 


0.4743  0.0132  0.0151 


0.75261  0.00211  0.0031 


0.1643 


0.8062 


0.2209 


0.3535 


0.0749 


0.0392 


0.1258 


0.3226 


0.0035  0.0031 


0.0024  0.0033 


0.0044  0.00461 


0.0049  0.0042 


1.30441 


0.1764 


0.2591 


1.7903 


1.1306 


0.0828 


0.0025 


0.01221  0.0117 


0.0131  0.0124 


0.0234 


0.0038 


0.0067  0.0077 


0.1095  0.0176 


0.6243  0.01971  0.0211 


0.2349 


■tnnEyiKBBgHI 


0.3033  0.00431  0.0045 


0.62791  0.0354 


1.0412 


8.2988  0.0158  0.0167 


0.0143 


0.0166 


0.0049 


0.0114 


0.0037 


0.0399 


0.0164 


0.0094 


0.0024 


0.0048 


0.0089 


0.0043 


0.0092 


0.0024 


0.0081 


0.0106 


0.0071 


0.0204 


0.0029 


0.0152 


0.0181 


0.014 


0.0145 


0.0133 


0.0263 


0.0028 


0.0068 


0.00251  0.0109 
074 


0.0 


0.005  0.0046 


0.0156  0.0054 


0.0094  0.0177 


0.0412  0.0095 


0.0112 


0.0012 


0.0065 


0.006 


0.0053 


0.0031 


0.008 


0.0009 


0.0013 


0.0077 


0.0046 


235 


0.002 


0.0024 
087 


0.0096 


0.009 


0.0037 


0.0162  0.0148 


0.0031  0.0036 


078 


0.013  0.0202 


0.019  0.0237 


0.02271  0.0198 


0.0034 


0.0045 


0.0362  0.0333 


0.0249 


0.0168 


Sample  Size  3 


Design  Point 
Reference  Number 
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Design  Point 
Reference  Number 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


51 


52 


53 


54 


55 


56 


57 


58 


59 


60 


61 


Smed 

Ethridge 

0.174 

0.1274 

0.2614 

0.2568 

0.242 

0.096 

0.1563 

0.1577 

0.093 

0.064 

0.1345 

0.1101 

0.087 

0.026 

0.164 

0.052 

0.3591 

0.1851 

Bilbbig 

0.1952 

0.1349 

0.0346 

ms 

0.1102 

0.3746 

0.1906 

0.3441 

0.3042 

0.0841 

0.081 

0.1161 

0.0684 

0.5603 

0.1308 

0.1979 

0.153 

0.1907 

0.1684 

MiliM 

0.1718 

0.1159 

0.1324 

0.1434 

0.0634 

0.0862 

0.1136 

0.054 

0.2533 

0.1946 

0.2947 

0.2132 

0.1293 

0.0943 

0.0881 

0.0443 

0.1357 

0.1567 

0.4187 

0.2297 

0.1713 

0.2228 

rarariaBiliEgH 

0.1347 

0.0334 

0.1854 

0.1085 

0.1277 

0.1163 

0.3336 

0.2215 

0.1301 

0.1463 

Grubbs  \Rayleigh\Numerical 


0.1235 


0.149 


0.1325 


0.1733 


0.2695 


0.0812 


0.0623 


0.041  0.0953 


0.0891 


622 


0.0558 


0.1458  0.1445  0.1364 


0.0699  0.0931  0.0764 


0.12 


0.2445 


0.2987 


0.1134 


0.045  0.0727 


0.2111 


0.0924 


0.2919  0.3547 


0.139 


ayiaaMijiigw 


0.054 


0.164 


0.1399 


0.2552 


prgwi] 

Jiam 


0.0896  0.1149 


0.0836 


0.0434 


0.1236 


0.1008 


0.11651 


0.15|  0.3043 


0.6617  0.23261 


0.11771  0.1241 


0.0462 


0.1435 


I 


0.1109 


0.2482 


0.2053 


0.1184 


0.0711  0.1047 


0.0392 


0.1185  0.1357 


.2598 


O 


0.81651  0.10481 


0.0555 


0.0671  0.0863 


0.1832 


0.1054 


0.1271 


0.2415 


0.1424 


0.0942 


0.0599 


0.0642 


0.1128 


0.1803 


0.1281 


0.0564 


0.1246 


0.2538 


0.29021 


0.0783 


0.0465 


0.2373 


0.19191 


0.2121 


0.0971 


0.108 


0.1062 


0.1027 


0.0584 


0.0613 


0.1067 


0.1889 


0.0995 


659 


0.0772 


0.0818 


0.0427 


0.1113 


0.2732 


0.2216 


0.0682 


0.0532 


0.0683 


0.142 


0.0978 


0.1794 


0.1007 


0.162|  0,1416 
1.2091  1.1352 


0.1483  0.0858 


0.1013  0.1099 


0.0657  0.0781 


0.1037 


0.0647 


0.0498  0.0233 


0.1049 


0.1913 


0.14951  0.1319 


0.256 


0.2626  0.: 


0.0878 


0.0578 


0.2518  0.1827 


0.1495  0.1044 


0.2388  0.1582 


0.1307 


0.0788  0.0879 


0.0633  0.0737 


0.2599  0.2326 


0.1938  0.1242 


0.1231 


0.0481 


0.1247 


0.2369  0. 


0.168 


0.0954 


0.138 


0.073 


0.0996  0.0715 
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Design  Point 
Reference  Number 


79 


80 


81 


82 


83 


84 


85 


86 


87 


88 
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0.1626 


0.172 


0.2478 


0.0417 


0.0581 


0.1627 


0.1452 


0.15 


0.1176 


0.0403 


0.3975 


0.0723 


0.1026 


0.0708 


0.0773 


0.0918 


0.0888 


0.1298 


0.0821 


0.0304 


0.0769 


0.0887 


0.0858 


0.0585 


0.0424 


0.0646 


0.1358 


0.1547 


0.1204 


0.263 


0.1305 


0.1994 


0.0786 


0.1183 


0.1312 


0.1281 


0.0613 


0.1273 


0.13761 


0.0433 


0.1124 


0.03 


0.0574 


0.0589 


PIO 

P^Jfawi 

p<8pz 


0.0616 


0.0846 


0.0629 


0.0778 


0.0439 


0.0376 


0.0834 


0.1327 


m 


0.1609 


0.1109 


0.0762 


0.1158 


0.0509 


0.0792 


0.0591 


0.098 


0.1078 


0.0339 


0.1088 


0.097 


0.1809 


0.0738 


0.1813 


0.0638 


0.0645 


0.1159 


0.0429 


0.0475 


0.0781 


0.066 


0.0514 


0.0776 


0.029 


0.0349 


0.0374 


0.0482 


0.055 


0.0404 


0.0597 


0.1749 


0.2314 


0.1765 


0.2174  0.1438 


0.1218  0.1076 


0.1051  0.06651 


0.1388  0.1093 


0.05581  0.05841 


0.0595  0.0473 


0.1224 


0.1344 


SI 


0.1291 


0.2376 


0.097 


0.2465 


0.0645 


0.0559 


0.1597 


0.0464 


0.0627 


0.0702 


0.0534 


0.0636 


0.079 


0.0555 


Sik) 


0.0269  0.0302 


0.058  0.0582 


0.0689  0.0526 


0.095  0.0724 


0.0661  0.061 


0.1481  0.1279 


0.3016  0.2024 


0.0948 


0.1051 


0.086 


0.0837 


0.0499 


0.0447 


0.1038 


0.0997 


0.0915 


0.1687 


0.073 


0.1914 


0.0529 


0.0572 


0.0682 


0.1273 


0.078 


0.1427 


0.034 


0.0542 


0.05341 


0.0673 


0.082 


0.0772 


0.0291 


0.0476 


0.0451 


0.0547 


502 


0.0701 


0.0554 


0.1194 


0.1475 


0.1138 


0.0455 


PIft«- 

PBI 

BE! 


BP 


0. 


0.0598 


0.0579 


0.0809 


0.1056 


0.0457  0.0452 


0.1026  0.1094 


0.1125 


0.0697 


0.1648 


0.0623 


0.1758  0.1401 


0.0448  0.0433 


BtiTa 


755 


322 


0.0565 


0.0919  0.1209 


0.0227  0.0215 


SBL 
SI 


SQ 

Slifail 

SI 


0.1851  0 


0.1514 


113 

0.1029 

0.0528 

0.0551 

0.0527 

0.0517 

0.0466 

114 

0.0691 

0.0912 

0.126 

■UIkIkI 

0.102 

0.083 

115 

0.0503 

0.0538 

0.0964 

0.0378 

0.0424 

116 

0.0506 

■lIlksKl 

0.9542 

0.0425 

0.057 

117 

0.2458 

0.2492 

0.2295 

118 

0.1417 

0.1244 

0.1969 

0.1774 
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157 
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0.0584 


0.015 


0.0244 


0.2205 


0.0409 


0.0127 


0.0172 


0.0244 


0.057 


0.1172 


0.0738 


0.0364 


0.0787 


0.1526 


0.1325 


0.0564 


0.0865 


0.0267 


0.0982 


0.0369 


0.0762 


0.1138 


0.0455 


0.2416 


0.0944 


0.158 


0.0298 


0.0793 


0.0131 


0.0565 


0.0287 


0.0449 


0.1201 


0.0188 


0.0328 


0.0393 


0.0511 


0.2312 


0.0214 


0.0325 


0.1274 


0.015 


0.0297 


0.0735 


0.0436 


0.0127 


0.0067 


0.0355 


0.4963  0.1435 


0.10161  0.0385 


0.086 


lEpi 


lilW 

iiiiiiAl] 


0.0157 


0.0656 


0. 


0.2955 


0.04421 


0.0756 


0.42441 


0.015 


0.0347 


0.0169 


0.0147 


0.0454 


0.077 


0.0785 


0.0367 


0.08 


0.1312 


0.1999 


0.0406 


0.0711 


0.0557 


0.1403 


0.8315 


0.1228 


0.47221 


0.0498 


5.3503 


0.06581 


0.0689 


29.377 


0.1361 


0.1316 


1.5615 


0.11561 


0.8593 


£ 

£ 

^H£ 

T 

A 


0.0403 


0.057  0.0563 


0.0253 


0.0345 


0.0548 


0.0731 


0.0243 


0.0299 


0.0817 


0.0172 


0.0488 


0.1048 


0.0188 


0.0362 


0.0888 


0.8067 


0.1484 


0.0814 


0.2921 


0.0159 


0.0292 


0.0102 


0.0574 


0.249 


0.0163 


0.0302 


0.0985 


0.0203 


0.0384 


0.0568 


IlllgfcVJMITOal 

HMH  miBilil  f  II 


239 


0.0873 


0.0362 


0.0139 


0.0088 


0.0389 


0.0252 


0.0439 


0.0572 


0.0155 


0.0592 


0.01271 


0.0303 


0.04661 


0.0249 


0.0235 


0.0289 


0.0422 


0.0232 


0.023 


0.0386 


0.0636 


0.0638 


0.0355 


0.0956 


0.078 


0.1756 


0.0632 


0.0113 


0.0235 


0.0363 


0.0664 


0.0177 


0.024 


0.0799  0.06 


0.0246 

■ataaia 

0.0142 

0.0037 

■amlAI 

0.0508 

0.0415 

0.0297 

0.0163 

0.0172 

0.1441 

0.1357 

0.04 

0.0535 

0.0471 

0.0558 

0.0227 

0.0191 

ike^eeebi 

0.0693 

0.0838 

0.0395 

0.0528 

0.0947 

0.0802 

0.2453 

0.2093 

0.0489 

0.0379 

0.009 


0.0174 


0.0267  0.0224 


0.0362  0.043 


254 

o 

o 

182 

0.01 

0.0158 


.053 


0.1698  0.1577 
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Design  Point 
Reference  Number 


199 


200 


201 


2 


2 


204 


205 


206 


207 


208 


209 


210 


211 


212 


213 


214 


215 


216 


7 


8 


219 


220 


221 


222 


223 


Smed  \Ethridge\MRand\Valstar  Grubbs \Rayleigh\Numerical 


0,0966 


0.139 


0.0216 


0.1022 


0.1331 


0.07 


0.1705 


0.0492 


0.1363 


0.1342 


0.2972 


0.0737 


0.0145 


0.0257 


0.0241 


0.0144 


0.0025 


0.0408 


0.0171 


0.027 


0.011 


0.0014 


0.0888 


0.0589 


0.0717 


0.0771 


0.0752 


0.016 


0.0476 


0.0949 


0.0825 


0.0235 


0.0683 


0.1304 


0.0521 


0.006 


0.0133 


0, 


0.106 


0.041 


0.0975 


0.1615 


1.7636 


0.0601 


0.1277 


0.1601 


0.1611 


0.4616 


3.598 


1.1297 


0.06131  0.05371 


267 


0.022  0.0171 


0.0497  0.0493 


0.0513 


0.0657 


0.0199 


0.0335 


0.0741 


0.1664  0.1221 


.0233 


0.15621  0. 


0.195  0.1718 


0.091  0.0797 


0.0286 


0.0781 


0.1184 


367 


0.0027  0.00261 


0.0311 


0.0098 


0.0418 


0.0026 


0.0016 


0.0604 


0.0333 


0.049 


16.124 


4.7478 


1.6913 


4.3592 


11.937 


83.651 


0.0212  0.0223 


0.0393 


0.0053 


0.0042 


0.0164 


10.014 


33.806 


0.0015  0.0017 


0.0337  0.041 


0.0145 


0.00161  0.0034 


0.01861 


0.0123 


0.0239 


0.0073 


0.012 


0.009 


0.0061 


0.0244 


0.0087 


0.0209 


3.86241  0.00311  0.0033 


0.0066  1.307 


0.007  13.678 


0.0195  237.321  0.0207|  0.0194| 


0.0065  0.3441 


0.0454 


0.0321 


0.0463 


0.0158 


0.0027 


0.012 


0.007 


0.0049 


0.0077 


0.0059 


0.0198 


0.0049 


0.0072 


0.0127 


0.0035 


0.0083 


0.0067 


0.0122 


0.0145 


0.0787 


0.08621  0.0881 


0.1543  0.1363 


.023 


0.0 


0.0113  0.0263 


0.0252  0.0432 


0.0164  0.0222 


0.00491  0.012 


0.0 


29  0.0701 
0.0331  0.0402 


0.0507  0.0493 


0.0434  0.0661 


0.5586  0.7791 


0.0073  0.0208 


0.4599  0.5901 


0.0043  0.008 


Design  Point 
Reference  Number 


239 


240 


241 


242 


243 


244 


245 


246 


247 


8 


249 


250 


251 


252 


1  253 


254 


Smed 

Ethridge 

MRcmd 

Valstar 

Grubbs 

Rayleigh . 

IMigli 

0.013 

■♦■aririsl 

0.0119 

0.0139 

0.0122 

0.0096 

mssuMA 

0.0115 

0.0105" 

0.0087 

WiIiVJMI 

0.0141 

lilclkHMwwamjwiiri 

0.0139 

0.0228 

0.0141 

12.573 

0.0172 

0.0279 

0.0205 

0.1492 

0.0182 

0.0175 

omii 

IBiligli 

0.0232 

0.7206 

0.0299 

0.0249 

0.0229 

8.2542 

0.0127 

0.0457 

mtsm 

4.4251 

0.0128 

0.0111 

0.0137 

0.021 

0.0389 

72.995 

0.0196 

0.0249 

0.0279 

0.0422 

0.0518 

10.336 

0.0436 

0.0217 

0.0296 

0.0285 

22.934 

0.0204 

0.0176 

0.0174 

53.814 

0.0114 

0.0119 

0.0091 

0.008 

0.0026 

0.3887 

0.0022 

0.004 

0.0053 

0.0049 

0.0039 

■tlWailMIMtllEtaMIKIIttEI 

0.0049 

0.0191 

0.0238 

16.016 

0.027 

0.0242 

0.0231 

0.0132 


0.0167 


0.019  0.027 


0.0266  0.0222 


0.07921  0.086 


0.025 


0.0542 


0.0097  0.0363 


0.0 


0.0033  0.0162 


0.0146 


0.0209 


0.0312 


0.0449 


0.0153 


0.0125 


0.0288 


0.0358 


0.0522 


0.019 


0.0179 


0.0294 


0.0109 


0.012 


0.0163 


0.02981 


0.0073 


0.0097 


0.0206 


0.0398 


0.0087 


0.0212 


0.0057 


0.0398 


0.041 


0.056 


0.005 


0.0136 


0.0356 


0.0641 


2.0432 


6.0896 


1.339 


15.56 


0.2154 


3.2053 


0.4653 


4.3134 


15.313 


2.048 


3.2741 


2.1793 


2.9216 


5.577 


12.718 


0.3651 


4.7282 


160.26 


0.0074 

0.0099 

0.0109 

0.0099 

0.0141 

0.0139 

0.017 

0.0175 

0.0352 

0.031 

0.0085 

0.0088 

0.0085 

0.0105 

0.0352 

0.0372 

0.0431 

0.0356 

0.0115 

0.0083 

0.0224 


0.0202 


0.0134 


0.011 


0.0174 


0.0223 


0.0066 


0.0122 


0.0187 


0.0423 


0.0281 


0.0121 


0.007 


0.0248 


0.014  0.0227 


0.0175 


26 

0.01 

77 

0.012  0.0232 


0.0087  0.0141 


0.0234  0.0278 


0.0352  0.052 


0.0668  0.0636 


0. 


0.0547 
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Design  Point 
Reference  Number 


1 


2 


3 


Appendix  H:  The  Sample  Analysis  Set  MSRE  Results 
Sample  Size  1 5 


Grubbs  \Rayleigh\Numerical 


Smed 

Ethridge 

0.0703 

0.0185 

0.041 

0.0238 

0.0226 

0.0158 

0.0337 

0.0174 

0.0221 

0.0081 

0.0525 

0.0318 

0.0197 

0.0152 

0.0193 

0.0166 

0.0209 

0.0151 

0.0316 

0.0134 

0.0473 

0.0302 

0.0351 

0.0275 

0.0135 

0.016 

0.0308 

0.0313 

0.0475 

0.0143 

0.0257 

0.0168 

0.0234 

0.0196 

0.0215 

0.0189 

0.0211 

0.0139 

0.0154 

0.0103 

0.0634 

0.0301 

0.038 

0.0248 

0.0274 

0.014 

0.0292 

0.0183 

0.0381 

0.0215 

0.0277 

0.0223 

0.019 

0.012 

0.0093 

0.0052 

0.0116 

0.0086 

0.0109 

0.0096 

0.0282 

0.0166 

0.0267 

0.0162 

0.0272 

0.0201 

0.0054 

0.0118 

0.029 

0.0095 

0.0196 

0.0132 

0.0115 

0.0066 

0.0187 


0.016 


0.0094 


0.0121 


0.0107 


0.0183 


0.0086 


0.0067 


0.0114 


0.015 


0.0203 


0.0149 


0.0092 


0.0196 


0.0106 


0.0072 


0.0132 


0.0096 


0.0116 


0.0097 


0.0157 


0.011 


0.0136 


Valstar 


0.0291 


0.0225 


0.0139 


0.0165 


0.0114 


0.0213 


0.0095 


0.0072 


0.0117 


0.026 


0.0192 


0.01 


0.011 


0.0109 


0.0221 


0.0077 


0.0068 


0.0113 


0.0274  0.0218 


0.0203 


0.0127 


0.0126 


0.0128 


0.0107 


0.0186 


0.0108 


0.0103 


0.013 


0.0091 


0.0125 


0.0267 


0.0144 


0.0134 


liisnaaiifitm 


0.0083 


0.0062 


0.0068 


0.0139 


0.0088 


0.0125 


0.014 


0.0043 


0.0184 


0.0156 


0.0069 


0.0074 


0.0088 


0.0069 


0.0203 


0.0115 


0.0153 


0.0108 


0.0211 


0.0131 


0.0127 


0.0132 


0.0181 


0.0158 


0.0065 


0.0049 


0.0028 


0.0135 


0.0081 


0.0091 


PUlweiiilgEgi 


0.007  0.006 


HIIMIKIIIIIIEKI 


0.0205  0.0113 


RlUfaDIiffiS 


0.0537 


0.0305 


0.0108 


0.0169 


0.02 


0.0393 


0.0083 


0.0077 


0.0103 


0.0244 


0.0326 


0.0148 


0.0087 


0.0193 


0.0232 


0.0242 


0.0149 


0.0095 


0.0108 


0.0124 


0.0471 


0.0249 


0.0158 


0.0167 


0.0315 


0.0273 


0.0131 


0.0032 


0.0046 


0.0204 


0.0167 


0.0121 


0.0141 


0.005 


0.0219 


0.0216 


0.0048 


0.0245  10.0225 


0.0184  0.0188 


0.0099  0.0103 


0.0104  0.0104 


WifiVifsiiiiia 


0.0095 


0.0131 


0.0091 


0.0105 


0.01 


0.0198 


0.012 


0.0125 


0.013 


0.0171 


0.007 


0.0114 


0.0145 


0.0216 


0.0144 


0.0094 


0.0186 


0.0072 


0.0111 


0.014 


0.0215 


0.0146 


0.0095 


0.0186 


0.0083 


0.0129 


0.0091 


0.0104 


0.01 


0.0177 


0.0112 


0.0121 


0.013 


0.0163 


0.0025  10.0026 


0.0097  0.009 


0.0068  0.0064 


0.0088  0.0085 


0.00 

65 

1 

EEl 

0.0 


0.0059  0.0057 
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Design  Point 
Reference  Number 


38 


39 


40 


41 


42 


Smed 

Ethridge  i 

0.0169 

0.0349 

0.0196 

0.0138 

0.0143 

0.0101 

0.0386 

0.0217 

0.0352 

0.0262 

0.018 

0.0212 

0.0314 

0.0174 

0.0182 

0.0105 

0.053 

0.0248 

0.0326 

0.028 

0.0154 

0.0075 

0.025 

0.0133 

0.0159 

0.0116 

0.0203 

0.0177 

0.0061 

0.005 

0.0069 

0.0056 

0.0062 

0.0038 

0.0105 

0.004  1 

0.0108 

0.0131 

0.0088 

0.0087" 

0.0114 

0.0093 

0.0089 

0.0098 

0.0084 

0.0056 

0.0162 

0.0173 

0.012 

0.0136 

0.0106 

0.0108 

0.0114 

0.0085 

0.0079 

0.0112 

0.0226 

0.012 

0.021 

0.0181 

0.0078 

0.0208 

0.0144 

0.0077 

0.0031 

0.0041 

0.0137 

0.0151 

0.0112 

0.0126 

0.0174 

0.0205 

0.0208 

0.0223 

0.0082 

0.0076 

0.0026 

0.0066 

0.0023 

0.0013 

0.0013 

0.0013 

Rayleigh \Numerical 


0.0101 


0.015 


riTwarjiiiii 


0.022710.0169 


0.0216  0.0136 


0.0302  0.0176 


0.0331  0.0182 


0.0625  0.033 


0.0208  0.0123 


0.02241  0.013 


0.015210.0117 


0.006 


0.0041 


0.005  I  0.004 


0.008 


0.005310.0041 


imuisttinMimaa 


0.0054 


0.0112 


0.0069  0.0068 


0.0184  0.0107 


niiiiaanniiiei 


0.0091 1  0.006 


0.0101  0.0063 


0.0072  0.0054 


0.0148  0.0115 


0.0163 


0.0088 


0.0185 


0.007 


0.012 


0.0142 


0.0218  0.0113 


0.0096  0.009 


0.01871  0.01 


0.0193 


0.025910.0304 


Bisncaiiiinsaiiigiiai 


0.0134 


0.0126 


0.0093 


0.0219 


0.0184 


0.0143 


0.0188 


0.0259 


0.0461 


0.0309 


0.0142 


0.0182 


0.0194 


0.0117 


0.0051 


0.0052 


0.0044 


0.006 


0.0071 


0.0048 


0.0086 


0.0091 


0.0089 


0.0079 


0.0076 


0.0067 


0.0055 


0.012 


0.0104 


0.0113 


0.0109 


0.0101 


0.0041 


0.0096 


0.0076 


0.0163 


0.01  I  0.01 


0.0165  0.0153 


0.007  0.0072 


lililTCTpIgtgi 


0.0136  0.0133 


0.0177  10.0166 


57 


0.0282  0.0253 


0.0191  0.0173 


0.1382  0.0035  0.0045 


0.0581 


0.0609  0.0013 


0.0095 


0.0053 


0.011 


0.0144 


0.01 


0.014 


0.0034  0.0034 


0.004  0.004 


0.0041  1 0.0 


0.0071 


0.0064  10.0067 


0.0066  0.0067 


0.0074  0.0068 


0.0093  0.0 


0.0065  0.0061 


0.0058  1 0.0055 


0.0 


0.0095  0.0093 


0.013  0.0115 


liltliBMPPM 


0.0092  0.0083 


anikBPPI 


0.0032  0.0033 


0.001  0.0009 
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Design  Point 
Reference  Number 

Smed 

Ethridge 

MRand 

Valstar 

Grubbs 

Rayleigh 

Numerical 

mCBN 

79 

[•Itltltgj 

0.0365 

0.0009 

0.0009 

0.003 

■IIIHUBMI 

80 

BES3I 

0.155 

0.004 

0.0036 

0.0039 

0.003 

0.0029 

81 

0.0026 

0.002 

0.1665 

0.0022 

0.0011 

0.0012 

82 

0.0022 

0.0022 

0.0392 

0.0012 

0.0012 

0.0037 

0.0012 

0.0015 

83 

0.0012 

0.0012 

0.012 

0.0013 

0.0011 

0.0029 

0.0012 

0.0015 

84 

0.0026 

0.0011 

0.0142 

0.0012 

0.0011 

0.0025 

iiiiniM 

85 

0.0016 

0.001 

0.0011 

0.0022 

86 

0.0034 

0.003 

0.0494 

0.0034 

0.0026 

0.0026 

87 

0.0016 

0.0025 

0.0045 

88 

0.0027 

0.0023 

0.0168 

0.0021 

0.002 

0.0043 

0.002 

0.0026 

89 

0.0036 

0.0024 

0.0232 

0.003 

0.0024 

0.0048 

0.0026 

0.0025 

90 

0.0033 

0.0019 

0.0692 

0.0027 

91 

0.0042 

0.0033 

0.1033 

0.002 

0.0019 

0.0046 

92 

0.0042 

0.0036 

0.0169 

0.0025 

0.0022 

0.0051 

0.0022 

0.0028 

93 

0.0018 

0.004 

0.0081 

0.0015 

0.0023 

0.0048 

0.002 

0.0034 

94 

0.0047 

0.0035 

0.0265 

0.0046 

0.0023 

0.0026 

95 

0.0027 

0.0035 

0.1121 

0.004 

0.0035 

0.0044 

96 

0.0046 

0.0026 

0.002 

0.003 

0.0072 

0.0053 

0.027 

0.0037 

0.0031 

0.0049 

0.0031 

0.0034 

0.0129 

0.0062 

0.0074 

0.0087 

0.0073 

99 

0.0043 

0.0049 

0.0322 

0.006 

0.0045 

0.0046 

100 

0.0019 

0.0029 

0.1441 

0.0042 

0.0034 

0.0029 

0.0028 

0.0026 
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Sample  Size  =  9 


Smed 

Ethridge  . 

0.067 

0.0312 

0.0493 

0.0237 

0.0322 

0.0162 

0.0453 

0.0408 

0.0713 

0.0346 

0.0434 

0.0247 

0.0265 

0.0395 

0.0131 

0.0114 

0.0313 

0.0151 

0.0238 

0.0162 

0.034 

0.0351 

0.0326 

EE^I 

0.0228 

0.0211 

0.0305 

0.0412 

0.0333 

0.0377 

0.0926 

0.0468 

0.0396 

0.0458 

0.0368 

0.0302 

0.0598 

0.0468 

0.0665 

0.0232 

0.1027 

0.0522 

0.0579 

0.0475 

0.0455 

0.0243 

0.0623 

0.0302 

0.0535 

0.0389 

0.0762 

0.0454 

0.0244 

0.0109 

0.0266 

0.0258 

0.0247 

0.0186 

0.0208 

0.012 

0.0397 

0.0141 

0.0211 

0.02 

0.0323 

0.0295 

0.0298 

0.0258 

0.0522 

0.0152 

0.0219 

0.0119 

0.0348 

0.0287 

0.0118 

0.0228 

0.0279 

0.0148 

0.0081 


0.0086 


0.0181 


0.02 


0.0163 


0.0213 


0.0047 


0.0122 


0.0153 


0.0224 


0.019 


0.0126 


0.0227 


0.0129  0.0139 


0.0224 


0.0241 


0.0186 


0.0221 


0.0219 


0.0047 


0.0134 


0.0174 


0.0262 


0.0196 


0.0211 


0.0047 


0.0128 


0.0149 


0.0209 


0.0175 


[■hwcmPB! 


0.0213  0.0289 


0.0257  0.0262 


0.0153 


0.0238  0.0258 


0.025  10.0311 


0.036 


0.0336 


0.0259 


0.0258 


iiMiiigsirawigai 


0.0247 


0.0302 


0.0136 


0.0443 


0.028 


0.0347 


0.0132 


0.0245 


0.0099 


0.01 


0.0245 


0.016 


0.0514 


0.0326 


0.0493 


0.015 


0.026 


0.0137 


0.0423 


0.0277 


0.0387 


0.0182 


0.013710.0105 


0.0203  0.0202 


0.0173  0.0189 


0.0215  0.0253 


0.0165  0.0251 


0.0165  0.02 


0.0096 


0.0147 


0.02 


0.0128 


0.0181 


0.0147 


0.0752 


0.0323 


0.009 


0.0317 


0.0378 


0.0305 


0.025 


0.0052 


0.0117 


0.0191 


0.0296 


0.0182 


0.014 


0.0276 


0.0451 


0.0447 


0.0242 


0.016 


0.0293 


0.0398 


0.082 


0.0473 


0.0183 


0.0472 


0.0394 


0.0552 


0.0164 


0.0209 


0.011 


0.0167 


0.0265 


0.0132 


0.0201 


0.017 


0.0243 


0.0206 


52  1 0.0 


29 


BIB 


0.0185  0.019 


OTtMCTlMK 

BWWclillg 


0.0207 


0.0048 


0.013 


0.021 1 


0.0047 


0.013 


0.0144  0.0158 


0.0208  0.0249 


lili: 

Bfi! 

EE 

BfftfliEI 


233  10.0229 
0.0236 


0.0244  10.0199 


0.0157  0.0161 


0.0289  0.0256 


0.025  0.019 


0.0135  0.0144 


0.0275 


0.0339 


0.0158 


0.0225 


0.0077 


0.0087 


q^SEEI 


5 


0.0288 


0.0312 


0.0163 


0.0252 


0.0084 


0.0086 


0.0123  0.0118 


0.0163  0.0142 


0.012  0.009 


UMiiiK^BIiIiBn 
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Design  Point 
Reference  Number 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


51 


52 


53 


54 


55 


56 


57 


58 


59 


60 


61 


62 


63 


64 


65 


66 


67 


68 


Smed  \Ethridge\MRand 


Rayleigh  \Numerical 


0.034 


0.0216 


0.0254 


0.0369 


0.0534 


0.0141 


0.0725 


0.0538 


0.0559 


0.051 


0.0542 


0.0194 


0.0186 


0.0082 


0.0074 


0.0423 


0.0247 


0.0187 


0.0058 


0.0163 


0.0137 


0.0222 


0.0097 


0.0222 


0.0402 


0.0117 


0.0463 


0.0426 


0.0232 


0.0457 


0.0402 


0.0344 


0.0349 


0.0404 


IE 


0.0045 


0.0036 


0.0047 


0.0022 


0.0096 


0.0246 


0.0208 


0.0221 


0.0309 


0.0386 


0.015 


0.0512 


0.0373 


0.0473 


0.0341 


0.0331 


■■Illiwsi 


0.0122 


0.0047 


0.0051 


0.0208 


0.0144 


0.0157 


0.0062 


0.0134 


09 


0.022 


0.0257 


0.0284 


0.0213 


0.0636 


0.0438 


0.0379 


0.0198 


0.0175 


0.019 


0.0099 


0.005 


0.0085 


0.0405 


0.0169 


0.0131 


0.0296 


0.037 


0.0352 


0.0118 


0.0157 


0.0227 


0.0237 


0.0395 


0.0253 


0.025 


0.0279 


0.0094 


0.0332 


0.0191 


0.0185 


0.0076 


0.0494 


0.0194 


0.0132 


latiBgg 


0.006 


0.0108 


0.0253 


0.01 


0.0238 


0.021 


0.0198 


0.0328 


0.0209 


0.0142 


0.027 


0.0341 


0.0274 


0.023 


0.0054 


0.0021 


0.0023 


0.0019 


0.0057 


0.017910.0217 


0.026510.0327 


0.0127 


0.0048  0.0071 


0.0157  0.022 


0.007 


0.0354 


0.0134 


0.0132 


0.0035 


0.017 


0.0193 


0.0147 


0.0324 


0.0264 


0.0175 


0.031 


0.0552 


0.0424 


0.0285 


0.0474 


0.0462 


0.0603 


0.178 


0.1197 


0.0342 


0.065 


0.2448 


0.0191 


0.0402 


0.0323 


0.0213 


0.0047 


0.0147 


0.0235 


0.0115 


0.0249 


0.0206 


0.0153 


HMHCiStS|pl[l)8Erj{ 


0.0662  0.0436 


0.0319  0.022 


0.0335 


0.0299 


0.0089 


0.0142 


0.0214 


0.0297 


0.0167 


0.0604 


0.0378 


0.0368 


0.0256 


0.0238 


0.0113 


0.0098 


0.0048 


0.0065 


0.0302 


0.0123 


0.0129 


0.0044 


0.0144 


0.017 


0.0138 


0.0039 


0.01 


0.021 


0.0114 


0.0221 


0.0178 


0.0154 


0.0247 


0.0348 


0.0223 


0.0232 


0.0368 


0.03 


0.0217 


0.0125 


0.0146 


0.023 


0.0233 


0.0183 


0.0641 


0.0341 


0.0232 


0.0158 


0.0144 


0.02 


0.0222 


0.0212 


0.0559 


0.0316 


0.0157 


0.0119 


0.0096 


0.0061 


0.0345 


2 


0.013 


0.0163 


0.01 


0.0099 


0.0 


0.0066 


0.0321 


0.013 


0.0161  0.0137 


0.0135  10.0161 


0.0244  0.0226 


0.0085  0.007 


0.0224  I  0.019 


Rill 

BfS! 


0.0398  10.0325 


0.0201  0.0182 


0.0042  0.0052 


ia£ 


IIW 


0.0527 


0.075 


0.0042 


0.0023 


wsmnmr 


0.0038 


0.0057 


0.0022  0.0031 
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0.006 


0.0045 


0.0017 


0.0118 


0.004 


0.0106 


0.0061 


0.0124 


0.0068 


0.0061 


0.0121 


0.0127 


0.0135 


0.0215 


0.0174 


0.0055 


0.0094 


0.0088 


0.0038 

MiIiieMtiXilfiea 

0.0029 

0.0 

033 

0.0029 

0.0025 

o 

o 

d 

0.0024 

0.0026 

0.0994 


0.1876  0.0039 


0.0053  10.0776  0.0063  0.0053 


0.008  0.0479  0.007210.0069 


iiWhtelliMngiiiiinrjiwiiiial 


0.0051  1 0.031 6 1 0.0038  0.0037 


0.0057  0.6476  0.0073  0.0067 


0.0069  0.132  10.0109 


wiinittCTiilttEBU 


0.0049 


0.0098 


0.0093 


0.0063 


0.006 


HiinKaiwuTaeii 


0.0068  10.0478 


iimiKilEEBI 


0.0112 


0.0066 


0.0939 

0.0116 

0.0099 

0.0096 

0.1442 

0.0045 

0.0032 

0.0043 

0.0689 

0.0118 

0.0099 

0.0096 

0.0334^ 

l^■^iiaa^lK^nri 

0.0099 

0.075 

0.0141 

0.012 

0.0105 

0.3309 

0.0154 

0.0127 

0.0101 

0.011 


0.0039 


0.0054 


0.007 


0.011 


0.0038 


0.006 


0.0083 


0.0091 


0.0094 


0.0142 


0.0101 


0.0033 


0.0101 


0.0121 


0.0047 


0.0053 


0.0109 

0.0045 

0.0065 

0.008 


liPUP^ 

BlilSI 


0.0137 


0.01 


0.0036 


0.0 


0.0106 


0.0111 
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Sample  Size  =  6 


Design  Point 
Reference  Number 


1 


2 


3 


Smed 

Ethridge  i 

0.0807 

0.0622  ( 

0.1057 

0.0588  ( 

0.0306 

0.027  ( 

0.0349 

0.0308 

0.0748 

0.0574 

0.0803 

0.0731 

0.0224 

0.0347 

0.0255 

0.0236 

0.013 

0.0236  ( 

0.029 

0.0146  ( 

0.0267 

0.0227  ( 

0.0352 

0.0363  1 

0.0121 

0.0064  1 

0.0524 

0.0745  1 

0.0676 

0.0527  1 

0.0907 

0.0701  1 

0.039 

0.0414  1 

0.0438 

0.0449  1 

0.0398 

0.0555  1 

0.11 

0.0867  1 

0.1346 

0.0704  ' 

0.0518 

0.0366 

0.059 

0.0431 

0.0816 

0.0647 

0.0728 

0.0532 

0.0681 

0.0542 

0.0222 

0.0228 

0.0123 

0.0243 

0.0229 

0.0197 

0.0763 

0.0384 

0.0208 

0.0215 

0.0243 

0.0215 

0.0038 

0.006 

0.025 

0.0199 

0.1182 

0.0887 

0.007 

0.0062 

0.0252 

0.0235 

0.0072 

0.0056 

0.0358 

0.0478 

Rayleigh  \Numerical 


0.0373 


niiMsrjiifigggi 


0.0564  0.051 


0.0357 


0.020410.0227 


0.014 


HUg&iaCEi! 


0.0702  0.0609 


0.0205  0.02 


liTiTgcyjnn] 

imsmmsm 


0.0172  0.0169 


0.0141  0.015 


0.0302  0.03 


0.0085  0.0068 


inmmrmmi 


0.032710.0325 


0.0344  0.0353 


0.0409  0.0412 


0.105410.0905 


0.0367  0.0357 


0.0416  0.0378 


0.0473  0.0416 


0.0429 


0.0166 


0.0183 


0.0101 


0.0447 


0.0103 


0.0065 


0.0169 


0.047910.0385 


0.0177  0.0147 


0.0142  0.017 


0.0092  0.0085 


0.0615  0.0456 


HiwcitaiiiiiBTa 


0.0126  0.0136 


HiMmiraiBiiga! 


0.0099 


0.0051 


0.0137 


0.017310.0136 


0.0662 


0.0098  0.0085 


0.0163 


0.0067  0.005 


0.0214  0.0126 


0.0727 


0.0542 


0.0285 


0.0215 


0.0764 


0.082 


0.0227 


0.0207 


0.0148 


0.0138 


0.0236 


0.0292 


0.005 


0.0464 


0.0436 


0.0712 


0.0322 


0.0377 


0.0439 


0.0889 


0.1205 


0.049 


0.0438 


0.0562 


0.0525 


0.0541 


0.0167 


0.0182 


0.0111 


0.0491 


0.0206 


0.0129 


0.0054 


0.0166 


0.0947 


0.0067 


0.0175 


0.0041 


0.0085 


0.0476 


0.0343 


0.0214 


0.058 


0.0205 


0.0222 


0.0125 


0.0168 


0.0142 


0.0301 


0.0364 


0.0328 

0.026 


0.0159 


0.0432 


\mm\ 


0.0235 


0.0136 


0.0155 


0.0117 


0.0302 


0.0398 


0.0369 


0.0395 


0.0459 


0.0327 


0.0351 


0.0412 


0.0535 


0.0868 


0.0352 


0.0381 


0.0401 


0.0395 


0.0528 


0.037  0.0342 


QE 
lili! 
BI! 


0.0355  0.0326 


0.0139  0.0134 


0.0083  10.0086 


0.0398  0.0342 


0.0187  0.02 


0.0055  0.0056 


0.0614  0.05 


0.0093  0.0124 
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Design  Point 
Reference  Number 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


51 


52 


53 


54 


55 


56 


57 


58 


59 


60 


61 


62 


63 


64 


65 


66 


67 


68 


69 


70 


71 


72 


Smed 


0.0331 


0.0681 


Valstar  Grubbs  \Rayleigh\Numerical 


0.0325 


0.0703 


0.0266 


0.0626 


0.0298 


0.0857 


0.0286 


0.0644 


0.0297 


0.0675 


0.0255  10.0261 


0.0607  0.0492 


0.0327 

0.0249 

EE3SI 

0.0229 

0.0374 

0.0352 

0.0178 

0.0198 

0.0177 

0.0238 

0.0174 

0.0533 

0.0393 

0.0227 

0.0311 

0.0235 

0.0319 

0.0217 

0.0679 

0.0349 

0.0331 

0.0534 

niiiqacwinicgal 

0.031 1 

0.0806 

0.0581 

0.0437 

0.0465 

0.0416 

0.0846 

0.0725 

0.0562 

0.0644 

0.0546 

0.0597 

0.0517 

0.0684 

0.0571 

0.042 

0.0481 

0.0336 

0.0536 

0.0441 

0.04 

0.0626 

0.0377 

0.0514 

0.033 

0.0316 

MtfeglJMilclcbl 

0.0454 

0.0341 

0.026 

0.029 

o 

o 

O 

0.0139 

0.0173 

0.0181 

istmsastssm 

0.0149 

0.0116 

0.0111 

0.0128 

0.0114 

0.011 

0.0124 

0.0109 

0.0105 

0.0059 

0.0058 

0.0074 

0.0056 

0.031 

0.0194 

0.0293 

0.035 

0.026 

0.0242 

0.0219 

0.0283 

0.0158 

0.0303 

0.037 

0.0258 

0.0194 

0.0237 

0.0064 

0.0044 

0.008 

0.0088 

0.0061 

0.0061 

0.0061 

0.0106 

0.0162 

0.0133 

0.0137 

0.0133 

0.0137 

0.0125 

0.0133 

0.014 

0.0122 

0.012 

0.0122 

0.0983 

0.0508 

0.081 

0.0935 

0.0704 

0.0628 

0.0688 

0.0204 

0.0191 

0.0125 

0.0148 

isciilEBlEa 

0.0101 

0.0204 

0.0199 

0.0162 

0.0182 

0.0154 

0.0154 

0.0156 

0.0103 

0.012 

0.0107 

0.0131 

0.0105 

0.0236 

0.0351 

0.0241 

0.0267 

0.0202 

0.0231 

0.0202 

0.0601 

0.0403 

0.0536 

0.0589 

0.0431 

0.04 

0.041 

0.037r 

0.0287 

0.0338 

0.0403 

0.0269 

0.023 

0.0241 

0.0153 

0.024 

0.02 

0.0247 

0.0173 

0.0142 

0.0174 

0.0535 

0.0521 

0.0377 

0.0307 

0.0384 

0.0408 

0.047 

0.0554 

0.0363 

0.0306 

0.0368 

0.0664 

0.082 

0.0909 

0.0627 

0.0566 

0.0637 

0.0422 

0.0401 

0.0579 

0.081 

0.047 

0.0372 

0.0408 

0.0394 

0.0306 

0.0486 

0.0566 

0.0396 

0.0326 

0.0391 

0.0658 

0.036 

0.0692 

0.077 

0.051 

0.0421 

0.0498 

0.0868 

0.054 

0.1117 

0.1217 

0.086 

0.0707 

0.0834 

0.0454 

0.0348 

0.0428 

0.0489 

0.0372 

0.0318 

0.0361 

0.0039 

0.0046 

0.5022 

0.0068 

0.0056 

0.0051 

0.0047 

0.0069 

0.0076 

0.0486 

0.007 

0.0074 

0.0097 

0.0074 

0.0028 

0.0029 

0.2378 

0.0027 

0.0027 

0.0048 

0.0028 

0.0088 

0.0081 

0.0874 

0.0091 

0.0087 

0.0088 

0.0085 

0.0295 

0.0231 

0.2918 

0.0355 

0.2162 

0.0208 


0.0286 


0.038 


0.044 


0.0309 


0.0211 


0.0258 


0.02 


0.0193 


0.0058 


0.0135 


0.0112 


0.0613 


0.0109 


0.0158 


0.0108 


0.02 


0.0356 


0.0207 


0.0161 

0.0326 

0.0302 


0.0282 


0.0335 


0.0381 


0.0637 
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Design  Point 
Reference  Number 

Smed 

Ethridge 

MRand 

Valstar 

Grubbs 

Rayleigh 

81 

0.0037 

0.0053 

0.0045 

0.0048 

0.0058 

0.0078 

82 

0.0047 

0.0034 

0.003 

0.0031 

0.0039 

0.0051 

83 

0.0098 

0.0064 

0.0064 

iiiingg 

84 

0.0053 

0.0042 

0.4705 

0.0046 

0.0045 

0.005 

0.0057 

HlllllSli 

85 

0.0248 

0.017 

0.2674 

0.0162 

0.0279 

86 

0.0128 

0.0137 

0.1927 

0.0164 

0.0144 

0.0131 

0.0176 

imsM 

87 

0.0133 

0.0081 

0.3237 

0.0121 

0.0097 

0.0087 

0.0128 

0.0138 

88 

0.0041 

0.0053 

0.285 

0.0045 

0.0046 

0.0072 

0.0057 

0.0062 

89 

0.055 

0.046 

1.8768 

0.0559 

0.0511 

0.0418 

0.0513 

0.0565 

90 

0.013 

0.0085 

0.1541 

0.0096 

0.0094 

91 

0.0115 

0.0069 

0.008 

0.0071 

0.0082 

0.0094 

92 

0.031 1 

0.0216 

1.849 

0.0251 

0.0229 

0.0229 

0.0232 

0.0266 

93 

0.0488 

0.0299 

0.554 

0.0407 

0.0352 

0.0295 

0.0364 

0.0341 

94 

0.0461 

0.0216 

0.4577 

0.0336 

0.0276 

0.0218 

0.0289 

0.0283 

95 

0.0135 

0.0128 

0.0156 

0.0169 

96 

0.0136 

0.0086 

0.2007 

0.0121 

0.0092 

0.0095 

0.0098 

0.0096 

97  n 

0.0305 

0.0181 

0.3483 

0.0299 

0.0226 

0.0192 

98 

0.0292 

0.0159 

0.1131 

0.0151 

0.0226 

0.018 

99 

0.0522 

0.0288 

0.1919 

0.0376 

0.033 

100 

0.0207 

0.0117 

2.7041 

0.0205 

0.0162 

0.0116 

0.0407 

0.0384 
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Sample  Size  =  3 


Design  Point 
Reference  Number 


1 


2 


3 


Smed  \Ethridge\MRand\Valstar  Grubbs  \RayIeigh\Numerical\TMCB. 


Smed 

Ethridge  . 

0.2196 

0.1244  i 

0.0989 

0.0648  1 

0.0341 

0.0476 

0.0289 

0.0251 

0.1026 

0.0729 

0.0546 

0.0298 

0.0042 

0.0063 

0.0063 

0.0021 

0.0149 

0.0101 

0.034 

0.0228 

0.041 1 

0.0362 

0.0196 

0.0121 

0.0001 

0.0009 

0.0548 

0.0464 

0.1191 

0.0754 

0.1359 

0.1379 

0.0258 

0.0247 

0.021 

0.0121 

0.0091 

0.0149 

0.0685 

0.0477 

0.3268 

0.1324 

0.2474 

0.1836 

0.0181 

0.0099 

0.1599 

0.152 

0.1206 

0.0937 

0.0983 

0.08 

0.0632 

0.0563 

0.0295 

0.0373 

0.0242 

0.0345 

0.1337 

0.0881 

0.0476 

0.0364 

0.0168 

0.014 

0.0018 

0.004 

0.007 

0.0053 

0.0084 

0.0088 

0.0374 

0.0219 

0.0147 

0.0075 

0.0079 

0.0212 

0.0015 

0.0632 

0.0171 


0.0531 


0.015 


0.187  0.1716 


0.0593  0.0614 


HlDCWSlHIlTcGni 


0.0204  0.02 


0.0799  0.073 


0.0274 


0.0102 


0.0479 


0.0546 


0.0695 


0.019 


0.0089 


0.0142 


0.0473 


0.1107 


0.1046 


0.0184 


0.1061 


0.1441 


0.0811 


0.0458 


0.032 


0.0198 


0.0805 


0.0359 


0.0083 


0.003 


0.005 


0.0084 


0.0123 


0.0036 


0.0104 


0.0322 


0.0032  0.003 


0.0081  0.0081 


0.009810.0107 


0.0001  0.0002 


0.0556  0.0541 


0.0688  0.056 


0.097  0.0865 


liiiwirapjtnail 

f>iiHK5inii!i5Eli 


0.01561  0.014 


0.0729 


0.1535 


0.0642 


0.0397 


0.0204 


0.0762 


0.0219 


0.0054 


0.0019 


0.0089 


0.0232 


0.0321 


0.0118 


0.0006 


0.0371 


0.0555 


0.1077 


0.02 


0.0111 


0.0123 


0.0069  0.0073 


0.0057  0.0053 


0.015  0.016 


0.057  0.0455 


0.0199  0.0208 


0.0003  0.0005 


0.0558  0.056 


0.0091  0.0106 


0.0142  0.0125 


0.1524  0.1433 


0.0166  0.0188 


0.1614 


0.0141 


0.1203 


0.1407 


0.0901 


IlIIMrailKWig 


0.0175  0.018 


0.1039  0.0826 


0.0399 


0.0514 


0.0332 


0.0231 


0.0868 


i.i.i.kMairggi 


0.0027  0.0026 


0.0097 


0.0036 


0.0354  0.034 


0.0176  0.0209 


nnicwsilifi|^|Minicii;'lnmcgBil 


0.021 1 
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Design  Point 
Reference  Number 

Smed 

Ethridge 

MRand 

Rayleigh 

Numerical 

TMCBN 

81 

0.0053 

0.0048 

0.6876 

0.0057 

0.0051 

0.0052 

0.0081 

0.0096 

82 

0.01 

0.0089 

0.752 

0.0086 

0.0086 

0.0088 

0.023 

83 

0.0161 

10.105 

0.0138 

0.0385 

84 

0.0176 

0.018 

0.0155 

■iiiwskI 

0.0384 

85 

0.0446 

0.041 

0.0464 

0.0355 

0.0708 

0.0771 

86 

0.0405 

0.0212 

15.649 

0.0272 

0.0238 

0.0218 

262.52 

87 

0.0052 

0.0047 

0.3963 

0.0051 

0.0079 

0.0097 

88 

0.0095 

0.0073 

0.0763 

0.0055 

89 

0.0203 

0.0239 

0.8131 

0.0218 

0.0239 

0.0285 

90 

0.1677 

0.0971 

0.2915 

0.0967 

0.1145 

0.1036 

91 

0.09 

0.0438 

0.0554 

0.0442 

0.0584 

0.0629 

92 

0.0664 

0.0382 

2.2842 

0.0515 

0.0464 

0.0322 

93 

0.0166 

0.0083 

0.5928 

0.0084 

0.0082 

0.0093 

0.0083 

0.0094 

94 

0.0451 

0.0575 

4.4111 

0.0646 

0.06 

0.0517 

0.0612 

0.0661 

95 

0.1095 

0.1068 

4.3785 

0.1326 

0.1183 

0.0969 

0.1275 

0.1383 

96 

0.0622 

■•ItEWli 

29.806 

0.0512 

0.0438 

0.0379 

104.088 

105.94 

97 

0.0493 

EE^ 

82.727 

0.0419 

0.0334 

0.0245 

98 

0.0523 

0.0353 

7.8147 

0.0303 

0.0418 

0.0384 

99 

0.0604 

0.0567 

10.058 

0.0515 

100 

0.0857 

0.0728 

3.815 

0.0999 

0.0842 

0.0671 
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Appendix  I:  Results  of  the  Simulation  Experiment  Based  on  the  Design  Points 


In  each  section,  the  tables  indicate  for  each  CEP  estimator  the  number  of  design 
points  that  had  the  best  (lowest)  value  for  the  MOEs  considered  in  this  study.  The 
estimator  with  the  best  performance  for  a  given  factor  level  in  the  tables  is  shaded  for 


quick  interpretation  of  the  results. 
OVERALL  RESULTS 


TMCBN 

MRE 

258 

Smed 

109 

Valstar 

102 

Numerical 

121 

Rayleigh 

159 

EtMdge 

172 

Grubbs 

75 

MRand 

114 

TMCBN 

VRE 

257 

Smed 

134 

Valstar 

105 

Numerical 

103 

Rayleigh 

181 

Ethridge 

217 

Grubbs 

100 

MRand 

119 

TMCBN 

MSRE 

288 

Smed 

no 

Valstar 

98 

Numerical 

124 

Rayleigh 

177 

Ethridge 

182 

Grubbs 

77 

MRand 

120 

MRand 
Qrabbs 
Ethridge 
Rayleigh 
Numerical 
Valstar 
Smed 
TMCBN 

0  50  100  150  200  250  300 
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DESIGN  POINT  MRE  RESULTS 


SAMPLE  SIZE  =  15 


SAMPLE  SIZE  =  9 


Overall 

Smed 

22 

Ethridge 

36 

MRand 

29 

Valstar 

27 

Grubbs 

21 

Rayleigh 

30 

Numerical 

47 

TMCBN 

69 

Bias 

0 

0.5ct 

la 

2a 

4a 

Smed 

0 

1 

2 

9 

10 

Ethridge 

0 

10 

10 

10 

6 

MRand 

5  ;• 

13 

g 

3 

0 

Valstar 

4 

5 

0 

5 

13 

Grubbs 

2 

3 

5 

2 

9 

Rayleigh 

2 

5 

10 

12 

1 

Numerical 

2 

8  1 

10 

12 

..  15 

TMCBN 

1 

20 
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DESIGN  POINT  MRE  RESULTS 
(continued) 
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DESIGN  POINT  VRE  RESULTS 


SAMPLE  SIZE  =  1 5  SAMPLE  SIZE  -  9 
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DESIGN  POINT  VRE  RESULTS 
(continued) 

SAMPLE  SIZE  =  6  SAMPLE  SIZE  =  3 
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DESIGN  POINT  MSRE  RESULTS 


SAMPLE  SIZE  =  15 


SAMPLE  SIZE  -  9 
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DESIGN  POINT  MSRE  RESULTS 
(continued) 

SAMPLE  SIZE  =  6  SAMPLE  SIZE  =  3 
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Appendix  J:  Results  of  the  Simulation  Experiment  Based  on  the  Sample  Analysis  Sets 


Like  Appendix  I,  in  this  appendix  the  tables  indicate  for  each  CEP  estimator  the 
number  of  design  points  that  had  the  best  (lowest)  value  for  the  MOEs  considered  in  this 
study.  The  estimator  with  the  best  performance  for  a  given  factor  level  in  the  tables  is 
shaded  for  quick  interpretation  of  the  results. 
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SAMPLE  ANALYSIS  SET 
MRE  RESULTS 

SAMPLE  SIZE  =  15  SAMPLE  SIZE  =  9 
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SAMPLE  ANALYSIS  SET 
MRE  RESULTS  (continued) 
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SAMPLE  ANALYSIS  SET 
VRE  RESULTS 
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SAMPLE  ANALYSIS  SET 
VRE  RESULTS  (continued) 

SAMPLE  SIZE  =  6  SAMPLE  SIZE  =  3 
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SAMPLE  ANALYSIS  SET 
MSRE  RESULTS 

SAMPLE  SIZE  =  1 5  SAMPLE  SIZE  =  9 
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SAMPLE  ANALYSIS  SET 
MSRE  RESULTS  (continued) 

SAMPLE  SIZE  =  6  SAMPLE  SIZE  =  3 
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